-- Logs begin at Thu 2022-08-11 09:07:17 UTC, end at Thu 2022-08-11 10:19:31 UTC. -- Aug 11 09:39:31.462071 dv-cinder-815660-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 11 09:39:31.469417 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 11 09:39:31.471423 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 11 09:39:31 2022] *** Aug 11 09:39:31.471423 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 11 09:39:31.471423 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 11 09:39:31.471423 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: nodename: dv-cinder-815660-9-iscsi Aug 11 09:39:31.471423 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: machine: x86_64 Aug 11 09:39:31.471423 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: clock source: unix Aug 11 09:39:31.471808 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: pcre jit disabled Aug 11 09:39:31.471808 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: detected number of CPU cores: 4 Aug 11 09:39:31.471808 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: current working directory: / Aug 11 09:39:31.471808 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: detected binary path: /usr/bin/uwsgi-core Aug 11 09:39:31.471808 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: your processes number limit is 47680 Aug 11 09:39:31.471808 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: your memory page size is 4096 bytes Aug 11 09:39:31.471808 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: detected max file descriptor number: 2048 Aug 11 09:39:31.471808 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: lock engine: pthread robust mutexes Aug 11 09:39:31.471808 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: thunder lock: enabled Aug 11 09:39:31.471808 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 11 09:39:31.472386 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 11 09:39:31.512511 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: Python main interpreter initialized at 0x557aee22a9c0 Aug 11 09:39:31.512511 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: python threads support enabled Aug 11 09:39:31.512511 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: your server socket listen backlog is limited to 100 connections Aug 11 09:39:31.512511 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: your mercy for graceful operations on workers is 90 seconds Aug 11 09:39:31.512861 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: mapped 403077 bytes (393 KB) for 2 cores Aug 11 09:39:31.512970 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: *** Operational MODE: preforking *** Aug 11 09:39:31.513265 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: *** uWSGI is running in multiple interpreter mode *** Aug 11 09:39:31.513265 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: spawned uWSGI master process (pid: 81625) Aug 11 09:39:31.513468 dv-cinder-815660-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 11 09:39:31.514331 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: spawned uWSGI worker 1 (pid: 81638, cores: 1) Aug 11 09:39:31.514331 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: spawned uWSGI worker 2 (pid: 81639, cores: 1) Aug 11 09:39:31.514551 dv-cinder-815660-9-iscsi devstack@keystone.service[81625]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 11 09:39:37.023533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 09:39:37.023533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: from cryptography.utils import int_from_bytes Aug 11 09:39:37.024330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 09:39:37.024330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: from cryptography.utils import int_from_bytes Aug 11 09:39:37.024809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 09:39:37.024809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: from cryptography.utils import int_from_bytes Aug 11 09:39:37.024915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 09:39:37.024915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: from cryptography.utils import int_from_bytes Aug 11 09:39:37.073866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 11 09:39:37.074044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 11 09:39:37.074127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 11 09:39:37.074236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] command line args: [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 11 09:39:37.074322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 11 09:39:37.074481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] command line args: [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 11 09:39:37.074568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 11 09:39:37.074672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ================================================================================ {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 11 09:39:37.074755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 11 09:39:37.074915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ================================================================================ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 11 09:39:37.075011 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] admin_token = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.075155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] config_dir = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.075245 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] admin_token = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.075346 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.075431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config_dir = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.075527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] config_source = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.075604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.075709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] debug = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.075798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config_source = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.075903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] debug = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.075984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.076166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.076262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.076328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.076472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.076559 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.076682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.076770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.076931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.077039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.077187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.077271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.077367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_config_append = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.077447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_config_append = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.077558 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.077631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.077695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_dir = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.077753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_dir = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.077869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_file = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.077941 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.078107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.078236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.078349 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.078432 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.078593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.078713 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.078837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.078922 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.078998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.079115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.079190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.079314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.079390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.079511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.079633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.079717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.079863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.079949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.080095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.080177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.080330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.080453 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.080561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.080745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.080877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.081006 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.081176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.081269 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.081350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.081429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.081794 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.081882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.081962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] publish_errors = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.082078 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] publish_errors = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.082193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.082271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.082436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.082513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.082634 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.082710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.082823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.082917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.083039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.083117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.083272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] standard_threads = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.083353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] standard_threads = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.083502 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.083589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.083744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.083828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.083904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.083973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.084080 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] use_journal = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.084167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_journal = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.084244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_json = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.084318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] use_json = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.084416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_stderr = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.084493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] use_stderr = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.084586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] use_syslog = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.084673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_syslog = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.084814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.084896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 09:39:37.085089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.085171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.085327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.085407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.085572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.085656 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.085807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.085883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.086052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.086140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.086290 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.086366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.086536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.086618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.086772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.086856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.087018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.087781 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.087860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.088033 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.088245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.088327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.088500 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.088576 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.088735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.088877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.088983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.089215 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.089306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.089481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.089565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.089734 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.089820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.089987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.090113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.090231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.090387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.090469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.090705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.090781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.090965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.091041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.091214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.091321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.091462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.091635 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.091712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.091908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.091989 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.092137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.092355 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.092440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.092604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.092710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.092855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.093069 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.093164 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.093336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.093411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.093582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.093659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.093829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.093961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.094066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.094275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.094352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.094530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.094614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.094766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.094928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.095005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.095165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.095247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.095400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.095484 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.095650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.095762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.095865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.095960 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.096104 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.096214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.096352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.096455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.096582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.096693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.096815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.096893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.097059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.097175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.097290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.097371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.097520 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.097601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.097753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.097878 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.097984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.098099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.098232 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.098316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.098445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.098531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.098674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.098754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.098908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.098993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.099136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.099213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.099365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.099444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.099595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.099680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.099832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.099909 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.100061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.100178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.100292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.100378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.100529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.100610 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.100768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.100849 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.100999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.101129 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.101262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.101345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.101492 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.101568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.101741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.101827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.101978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.102062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.102214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.102298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.102449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.102539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.102683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.102802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.103487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.103570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.103725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.103804 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.103974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.104053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.104214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.104292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.104453 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.104531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.104700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.104780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.104937 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.105035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.105201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.105279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.105437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.105517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.105673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.105760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.105907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.105992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.106156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.106248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.106395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.106471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.106631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.106714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.106869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.106955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.107104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.107182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.107340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.107417 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.107572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.107661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.107807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.107885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.108042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.108120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.108301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.108387 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.108539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.108616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.108774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.108852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.109006 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.109142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.109268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.109345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.109501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.109577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.109734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.109819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.109971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.110048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.110212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.110293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.110431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.110517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.110663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.110741 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.110896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.110973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.external = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.111125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.111211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.111367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.111447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.111603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.111681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.111851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.111938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.password = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.112082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.token = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.112159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.112323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.112401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.112560 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.112647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.112812 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.112898 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.113059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.113141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.external = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.113297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.113376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.113543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.113632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.113776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.113855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.114012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.114128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.password = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.114261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] auth.token = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.114338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.114479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.114565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.114715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.114793 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.114952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.115038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.115179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.115257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.115420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.115496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.115665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.115743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.115898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.115983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.116134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.116212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.116385 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.116465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.116634 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.116714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.116881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.116960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.117141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.117224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.117386 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.117465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.117623 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.117708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.117876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.117954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.118118 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.118197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.118390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.118548 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.118779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.118862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.119034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.119112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.119284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.119363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.119524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.119603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.119779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.119858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.120019 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.120110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.120254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.120328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.120496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.120574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.120742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.120819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.120977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.121076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.121231 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.121310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.121480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.121558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.121714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.121793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.121949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.122028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.122186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.122263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.122416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.122494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.122655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.122734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.122902 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.122980 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.123137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.123223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.123370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.123447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.123600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.123678 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.123829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.123908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.124060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.124138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.124304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.124390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.124533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.124611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.124771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.124849 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.125034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.125128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.125290 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.125370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.125524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.125602 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.125755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.125847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.125986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.126066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.126222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.126300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.126454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.126540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.126690 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.126769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.126931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.127018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.127173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.127260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.127422 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.127500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.127653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.127732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.127897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.128024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.128129 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.128210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.128365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.128449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.128594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.128683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.128828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.128907 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.129086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.129168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.129328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.129419 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.129559 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.129637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.129792 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.129958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.130045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.130207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.130287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.130449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.130527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.130687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.130765 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.130927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.131006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.131177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.131256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.131412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.131489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.131643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.131723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.131873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.131950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.132114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.132191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.132354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.132434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.132593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.132706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.132850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.132949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.133105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.133182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.133308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.133389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.133549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.133628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.133777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.133857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.134056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.134178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.134297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.134377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.134536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.134655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.134766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.134853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.135000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.135080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.135232 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.135312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.135473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.135553 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.135713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.135799 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.135951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.136029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.136181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.136260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.136427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.136514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.136669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.136747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.136903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.136983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.137158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.137251 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.137394 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.137473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.137626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.137705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.137862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.137950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.138091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.138170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.138328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.138405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.138568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.138655 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.138805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.138884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.139046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.139130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.139289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.139376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.139519 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.139598 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.139764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.139844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.139992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.140078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.140226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.140304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.140460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.140589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.140692 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.140779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.140928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.141006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.141199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.141278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.141432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.141594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.141670 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.141740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.141884 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.141963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.142123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.142214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.142356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.142434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.142589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.142668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.142826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.142913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.143073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.143152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.143314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.143393 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.143545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.143631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.143778 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.143856 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.144016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.144093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.144255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.144341 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.144490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.144568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.144721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.144799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.144953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.145110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.145228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.145308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.145478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.145557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.145725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.145812 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.145971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.146050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.146207 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.146284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.146446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] role.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.146533 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.146690 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.146768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] role.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.146919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.146997 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.147155 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.147243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.147402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.147481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.147646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.147724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.147892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.147970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.148135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.148212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.148382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.148518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.148630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.148869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.148976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.149121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.149201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.149371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.160451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.160572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.160655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.160716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.160786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.160786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.160911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.160985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.161965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162393 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162532 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162678 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.162967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163044 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163393 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163681 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.163968 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.164961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165046 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165419 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165487 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165728 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165922 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.165994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.connection = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166557 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166638 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166788 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.166931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.167955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.168025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.168101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.168181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.168181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 11 09:39:37.168299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.168299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.168299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.168299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.168299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.168299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.168299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 09:39:37.168299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81639) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 11 09:39:37.192739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.192887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.192976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.193139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.193228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.193369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.203034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.203250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.232046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.232272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.232573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.232806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.284562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.285344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.334836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.335065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.335387 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.335628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.399414 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.400176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.400403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.400519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.400626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.400766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.400888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.400972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.401091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.401229 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.401455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.401673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.423720 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.423946 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.424049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.424171 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.424253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.424392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.424473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.424610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.424701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.424825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.424931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.425083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.425174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.425268 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.436206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.436531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.449582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.449816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.450040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.450126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.450262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.450345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.450480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.450572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.450702 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.450785 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.450917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81639) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.451000 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.451140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.451356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 09:39:37.467667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.467879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 09:39:37.468229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.468311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.468431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 09:39:37.468515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 09:39:37.468709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.468791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.468973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.469080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 09:39:37.469260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.469345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.469518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.469600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.469776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.470035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.481432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.481712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.481985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.482067 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.482266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.482361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.482542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.482829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.489153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.489786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.492201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.492491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.492784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.492865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.493093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.493200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.493481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.493571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.493759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.493877 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.494046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.494153 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.494320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.494401 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.494594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.494674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.494882 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.494963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.495160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.495241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.495459 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.495729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.518839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.519069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.520337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.520554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.520636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.520840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.525320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.525607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.525692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.525895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.525992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.526202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.532432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.532721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.532901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.533221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.536775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.537047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.537317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.537399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.537611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.537705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.537889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.537971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.538167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.538250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.538444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.538527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.538719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.538799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.538995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.539267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.554822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.555101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.556112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.556399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.559132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.560324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.562318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.562617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.563481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.563783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.566996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.568451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.570449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.572181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.574257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.574564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.574856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.575155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.575442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.575538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.575726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.575819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.576026 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.576317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.576601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.576903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.591739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.592028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.592305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.592571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.593195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.593500 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.593804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.594102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.599449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.600972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.601088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.601298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.602563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.603395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.604680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.604967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.605298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.605592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.605878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.606187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.606475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.606769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.607064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.607345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.617231 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.617518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.617805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.618094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.618456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.618732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.618831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.619011 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.619095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.619336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.619620 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.619906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.620186 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.620477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.636039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.637561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.639015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.640501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.641964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.643464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.643554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.644990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.645100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.645311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.645604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.646549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.646835 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.647135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.652042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.653679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.653768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.655318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.656603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.656899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.657215 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.657496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.658260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.658556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.658853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.659137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.666033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.666399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.666845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.667747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.668044 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.668329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.674154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81639) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.674443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.674728 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.675012 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.675223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 09:39:37.675305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.675512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.675603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.675802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.675952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.676078 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.676160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.676366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.676447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.676652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.676733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.676931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.677031 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.677230 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.677313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.677555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.677635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.677826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81639) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.677915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.678154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.678430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.678702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.678968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 09:39:37.712119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557aee22a9c0 pid: 81639 (default app) Aug 11 09:39:37.713532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557aee22a9c0 pid: 81638 (default app) Aug 11 09:39:37.715430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64de00-948c-4153-8cb1-c1f00bd985cf None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:37.715714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64de00-948c-4153-8cb1-c1f00bd985cf None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:37.715984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64de00-948c-4153-8cb1-c1f00bd985cf None None] PATH_INFO: `/v3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:37.720314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Thu Aug 11 09:39:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:39:46.208639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637352b-d4d0-4806-a7fd-0c916e4ea41a None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:46.209095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637352b-d4d0-4806-a7fd-0c916e4ea41a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:46.209221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637352b-d4d0-4806-a7fd-0c916e4ea41a None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:46.214338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:39:46.219451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006ef6d-069b-49b3-a265-09a24af48179 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:46.219806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006ef6d-069b-49b3-a265-09a24af48179 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:46.220001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006ef6d-069b-49b3-a265-09a24af48179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:46.290996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: INFO dbcounter [None req-d006ef6d-069b-49b3-a265-09a24af48179 None None] Registered counter for database keystone Aug 11 09:39:46.309963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG oslo_db.sqlalchemy.engines [None req-d006ef6d-069b-49b3-a265-09a24af48179 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81639) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 11 09:39:46.316076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writer thread running {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 11 09:39:46.494147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG passlib.handlers.bcrypt [None req-d006ef6d-069b-49b3-a265-09a24af48179 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81639) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 11 09:39:46.494491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG passlib.handlers.bcrypt [None req-d006ef6d-069b-49b3-a265-09a24af48179 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81639) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 11 09:39:46.548769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d006ef6d-069b-49b3-a265-09a24af48179 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:46.550415 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d006ef6d-069b-49b3-a265-09a24af48179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:46.678970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 11 09:39:46.782580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637352b-d4d0-4806-a7fd-0c916e4ea41a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:46.851176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: INFO dbcounter [None req-6637352b-d4d0-4806-a7fd-0c916e4ea41a None None] Registered counter for database keystone Aug 11 09:39:46.872765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG oslo_db.sqlalchemy.engines [None req-6637352b-d4d0-4806-a7fd-0c916e4ea41a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81638) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 11 09:39:46.878874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writer thread running {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 11 09:39:46.996093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637352b-d4d0-4806-a7fd-0c916e4ea41a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:47.000334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adacb4e-5609-4d88-a7bb-51bda2769090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:47.000947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adacb4e-5609-4d88-a7bb-51bda2769090 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:47.001320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adacb4e-5609-4d88-a7bb-51bda2769090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:47.001594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adacb4e-5609-4d88-a7bb-51bda2769090 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:47.009597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adacb4e-5609-4d88-a7bb-51bda2769090 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:47.061810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2adacb4e-5609-4d88-a7bb-51bda2769090 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 11 09:39:47.062095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adacb4e-5609-4d88-a7bb-51bda2769090 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:47.072224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Aug 11 09:39:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 09:39:47.078146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006ef6d-069b-49b3-a265-09a24af48179 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:47.092795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006ef6d-069b-49b3-a265-09a24af48179 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:47.097164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedc2d5-9b3a-476d-94c1-4bc7488345af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:47.098273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedc2d5-9b3a-476d-94c1-4bc7488345af admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:47.098551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedc2d5-9b3a-476d-94c1-4bc7488345af admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:47.098821 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedc2d5-9b3a-476d-94c1-4bc7488345af admin admin] PATH_INFO: `/v3/projects/c055f43eb0f74f81a1a2e702ea1b7964` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:47.100236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fedc2d5-9b3a-476d-94c1-4bc7488345af admin admin] RBAC: Authorizing `identity:get_project(project_id=c055f43eb0f74f81a1a2e702ea1b7964)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:47.150351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1fedc2d5-9b3a-476d-94c1-4bc7488345af admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 11 09:39:47.150688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fedc2d5-9b3a-476d-94c1-4bc7488345af admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:47.155265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Thu Aug 11 09:39:47 2022] GET /identity/v3/projects/c055f43eb0f74f81a1a2e702ea1b7964 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:39:49.117522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48733c0-5b4b-412c-9bd5-5f3d3243b5fc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:49.118068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48733c0-5b4b-412c-9bd5-5f3d3243b5fc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:49.118155 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48733c0-5b4b-412c-9bd5-5f3d3243b5fc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:49.122680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:39:49.128133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99d75-998f-4246-a0a5-7b15ae4e827b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:49.128565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99d75-998f-4246-a0a5-7b15ae4e827b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:49.128682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99d75-998f-4246-a0a5-7b15ae4e827b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:49.150344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-dec99d75-998f-4246-a0a5-7b15ae4e827b None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:49.151867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dec99d75-998f-4246-a0a5-7b15ae4e827b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:49.159674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 11 09:39:49.181873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48733c0-5b4b-412c-9bd5-5f3d3243b5fc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:49.194243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48733c0-5b4b-412c-9bd5-5f3d3243b5fc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:49.198463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e1cdd-9b0c-4ece-b621-49b6e986ca46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:49.199056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e1cdd-9b0c-4ece-b621-49b6e986ca46 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:49.199335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e1cdd-9b0c-4ece-b621-49b6e986ca46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:49.199604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e1cdd-9b0c-4ece-b621-49b6e986ca46 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:49.207170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e1cdd-9b0c-4ece-b621-49b6e986ca46 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:49.208924 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e1cdd-9b0c-4ece-b621-49b6e986ca46 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:49.218015 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Aug 11 09:39:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 09:39:49.223840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99d75-998f-4246-a0a5-7b15ae4e827b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:49.233581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99d75-998f-4246-a0a5-7b15ae4e827b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:49.237969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8206f51-a90b-4063-9220-f9dc42fd20a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:49.238588 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8206f51-a90b-4063-9220-f9dc42fd20a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:49.238870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8206f51-a90b-4063-9220-f9dc42fd20a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:49.239148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8206f51-a90b-4063-9220-f9dc42fd20a2 admin admin] PATH_INFO: `/v3/users/7936a8595af344cb984e181520011d53` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:49.240463 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8206f51-a90b-4063-9220-f9dc42fd20a2 admin admin] RBAC: Authorizing `identity:get_user(user_id=7936a8595af344cb984e181520011d53)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:49.242306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8206f51-a90b-4063-9220-f9dc42fd20a2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:49.244165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Thu Aug 11 09:39:49 2022] GET /identity/v3/users/7936a8595af344cb984e181520011d53 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:39:51.304229 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd55f03-6781-4ee5-8b8c-138dbcad33a8 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:51.304796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd55f03-6781-4ee5-8b8c-138dbcad33a8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:51.304796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd55f03-6781-4ee5-8b8c-138dbcad33a8 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:51.306650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:39:51.311651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38384e8d-d158-40e0-962a-508808431801 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:51.312067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38384e8d-d158-40e0-962a-508808431801 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:51.312321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38384e8d-d158-40e0-962a-508808431801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:51.331015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-38384e8d-d158-40e0-962a-508808431801 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:51.332186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-38384e8d-d158-40e0-962a-508808431801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:51.351238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb8fee-dcd3-4ec4-b8e7-bc81e13ceda3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:51.351528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb8fee-dcd3-4ec4-b8e7-bc81e13ceda3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:51.351809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb8fee-dcd3-4ec4-b8e7-bc81e13ceda3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:51.353210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:39:51.358008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311779f0-00db-4ea1-9707-dff09f95a008 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:51.358290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311779f0-00db-4ea1-9707-dff09f95a008 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:51.358565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311779f0-00db-4ea1-9707-dff09f95a008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:51.402968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG passlib.handlers.bcrypt [None req-311779f0-00db-4ea1-9707-dff09f95a008 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81638) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 11 09:39:51.403386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG passlib.handlers.bcrypt [None req-311779f0-00db-4ea1-9707-dff09f95a008 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81638) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 11 09:39:51.412460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:39:51.436058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38384e8d-d158-40e0-962a-508808431801 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:51.449436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-311779f0-00db-4ea1-9707-dff09f95a008 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:51.451117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-311779f0-00db-4ea1-9707-dff09f95a008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:51.459102 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:51.539561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38384e8d-d158-40e0-962a-508808431801 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:51.551853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:39:51.573409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311779f0-00db-4ea1-9707-dff09f95a008 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:51.574088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537831d-bbfb-48b6-8e38-debc3bba42c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:51.574728 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537831d-bbfb-48b6-8e38-debc3bba42c4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:51.575013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537831d-bbfb-48b6-8e38-debc3bba42c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:51.575314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537831d-bbfb-48b6-8e38-debc3bba42c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:51.586907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537831d-bbfb-48b6-8e38-debc3bba42c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-5537831d-bbfb-48b6-8e38-debc3bba42c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:39:51.596224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:39:51.597142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:39:51.597974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:39:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:39:51.603997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537831d-bbfb-48b6-8e38-debc3bba42c4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:51.676544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311779f0-00db-4ea1-9707-dff09f95a008 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:51.678833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:51.685907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:51.704438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537831d-bbfb-48b6-8e38-debc3bba42c4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:51.715619 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa51ab-7ca3-4734-a487-2a1fb43392cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:51.716496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa51ab-7ca3-4734-a487-2a1fb43392cf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:51.716779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa51ab-7ca3-4734-a487-2a1fb43392cf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:51.717077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa51ab-7ca3-4734-a487-2a1fb43392cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:51.738613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06e456-fe88-4b0b-8a71-5e8e79efca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:51.739216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e456-fe88-4b0b-8a71-5e8e79efca0d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:51.739496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e456-fe88-4b0b-8a71-5e8e79efca0d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:51.739768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06e456-fe88-4b0b-8a71-5e8e79efca0d None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:51.740436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06e456-fe88-4b0b-8a71-5e8e79efca0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:39:51.740746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06e456-fe88-4b0b-8a71-5e8e79efca0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:51.742168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06e456-fe88-4b0b-8a71-5e8e79efca0d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:51.753752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:39:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:39:51.759683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06e456-fe88-4b0b-8a71-5e8e79efca0d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:51.787732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa51ab-7ca3-4734-a487-2a1fb43392cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:51.789284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa51ab-7ca3-4734-a487-2a1fb43392cf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:51.891288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:39:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 09:39:51.897137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa51ab-7ca3-4734-a487-2a1fb43392cf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:51.900904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:51.968806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06e456-fe88-4b0b-8a71-5e8e79efca0d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:51.998683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa51ab-7ca3-4734-a487-2a1fb43392cf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:52.002303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75353ad-dbcf-461c-9ffc-8c209049509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:52.002938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75353ad-dbcf-461c-9ffc-8c209049509e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:52.003224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75353ad-dbcf-461c-9ffc-8c209049509e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:52.003501 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75353ad-dbcf-461c-9ffc-8c209049509e None admin] PATH_INFO: `/v3/users/7936a8595af344cb984e181520011d53` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:52.004645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75353ad-dbcf-461c-9ffc-8c209049509e None admin] RBAC: Authorizing `identity:get_user(user_id=7936a8595af344cb984e181520011d53)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:52.006152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75353ad-dbcf-461c-9ffc-8c209049509e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:52.007700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:39:51 2022] GET /identity/v3/users/7936a8595af344cb984e181520011d53 => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:39:52.014056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75353ad-dbcf-461c-9ffc-8c209049509e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:52.030141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:52.033429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55438c75-73d6-430e-a1f1-0c48dbe1ff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:52.034276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55438c75-73d6-430e-a1f1-0c48dbe1ff55 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:52.034549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55438c75-73d6-430e-a1f1-0c48dbe1ff55 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:52.034822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55438c75-73d6-430e-a1f1-0c48dbe1ff55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:52.053752 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55438c75-73d6-430e-a1f1-0c48dbe1ff55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:52.055697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55438c75-73d6-430e-a1f1-0c48dbe1ff55 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-55438c75-73d6-430e-a1f1-0c48dbe1ff55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:39:52.074679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:39:52.075574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:39:52.076604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:39:52.076604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:39:52.076604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:39:52.076604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:39:52.081525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55438c75-73d6-430e-a1f1-0c48dbe1ff55 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:52.116660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75353ad-dbcf-461c-9ffc-8c209049509e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:52.137003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:52.150650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4628a5-c704-440c-a182-e9754e2157cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:52.151279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4628a5-c704-440c-a182-e9754e2157cb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:52.151559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4628a5-c704-440c-a182-e9754e2157cb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:52.151833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4628a5-c704-440c-a182-e9754e2157cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:52.152940 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4628a5-c704-440c-a182-e9754e2157cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:52.154392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4628a5-c704-440c-a182-e9754e2157cb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:52.155924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:39:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:39:52.161780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4628a5-c704-440c-a182-e9754e2157cb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:52.179908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55438c75-73d6-430e-a1f1-0c48dbe1ff55 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:52.216335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b47940-f63b-4c00-88dd-d6b6100b7e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:52.216977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b47940-f63b-4c00-88dd-d6b6100b7e41 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:52.217369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b47940-f63b-4c00-88dd-d6b6100b7e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:52.217661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b47940-f63b-4c00-88dd-d6b6100b7e41 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:52.218355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b47940-f63b-4c00-88dd-d6b6100b7e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:39:52.218674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b47940-f63b-4c00-88dd-d6b6100b7e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:52.220163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b47940-f63b-4c00-88dd-d6b6100b7e41 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:52.247629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:39:52.247871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:52.269134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4628a5-c704-440c-a182-e9754e2157cb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:52.305039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52d341-6c71-462c-a242-22b1721487f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:52.306219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52d341-6c71-462c-a242-22b1721487f7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:52.306549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52d341-6c71-462c-a242-22b1721487f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:52.306860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52d341-6c71-462c-a242-22b1721487f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:52.307612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52d341-6c71-462c-a242-22b1721487f7 None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.domain.id=default, user.id=7936a8595af344cb984e181520011d53) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:39:52.307926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52d341-6c71-462c-a242-22b1721487f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:52.309430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52d341-6c71-462c-a242-22b1721487f7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:52.317042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Aug 11 09:39:52 2022] GET /identity/v3/role_assignments?user.id=7936a8595af344cb984e181520011d53&scope.domain.id=default&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:39:54.605899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43042f17-7d6c-46ae-9d41-69c242f0e21b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:54.606400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43042f17-7d6c-46ae-9d41-69c242f0e21b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:54.606505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43042f17-7d6c-46ae-9d41-69c242f0e21b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:54.607793 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:39:54.613081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2181d-7385-4f7e-a73c-91e656ca6926 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:54.613875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2181d-7385-4f7e-a73c-91e656ca6926 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:54.614354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2181d-7385-4f7e-a73c-91e656ca6926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:54.646311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f8a2181d-7385-4f7e-a73c-91e656ca6926 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:54.647858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f8a2181d-7385-4f7e-a73c-91e656ca6926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:54.738886 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 12/23] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:39:54.772245 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43042f17-7d6c-46ae-9d41-69c242f0e21b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:54.816232 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bafa8a-e705-401e-9354-97b0f8204062 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:54.816881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bafa8a-e705-401e-9354-97b0f8204062 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:54.817738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bafa8a-e705-401e-9354-97b0f8204062 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:54.819594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 13/24] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:39:54.825930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9a394-e220-4f8a-967d-d7f7239fa142 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:54.826460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9a394-e220-4f8a-967d-d7f7239fa142 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:54.826948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9a394-e220-4f8a-967d-d7f7239fa142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:54.866144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4de9a394-e220-4f8a-967d-d7f7239fa142 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:54.867750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4de9a394-e220-4f8a-967d-d7f7239fa142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:54.955267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43042f17-7d6c-46ae-9d41-69c242f0e21b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:54.990333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:55.006079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 14/25] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:39:55.007749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21f4f6-3810-469e-94eb-a1e691da13de None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:55.017379 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21f4f6-3810-469e-94eb-a1e691da13de None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:55.017927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21f4f6-3810-469e-94eb-a1e691da13de None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:55.028227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 15/26] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:55 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:39:55.028227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329e893-772b-4d80-a9a4-b84cad943f17 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:55.028227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329e893-772b-4d80-a9a4-b84cad943f17 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:55.028227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329e893-772b-4d80-a9a4-b84cad943f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:55.035419 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:55.042332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b7eb0-8ca6-4667-b455-e8258ce5ea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:55.042999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7eb0-8ca6-4667-b455-e8258ce5ea50 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:55.043286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7eb0-8ca6-4667-b455-e8258ce5ea50 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:55.043565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7eb0-8ca6-4667-b455-e8258ce5ea50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:55.048297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f329e893-772b-4d80-a9a4-b84cad943f17 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:55.049881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f329e893-772b-4d80-a9a4-b84cad943f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:55.073623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b7eb0-8ca6-4667-b455-e8258ce5ea50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:39:55.173111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 16/27] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:39:55.173939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329e893-772b-4d80-a9a4-b84cad943f17 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2a9b7eb0-8ca6-4667-b455-e8258ce5ea50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:39:55.189058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:39:55.189862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:39:55.190596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 12/28] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:39:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:39:55.206025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b7eb0-8ca6-4667-b455-e8258ce5ea50 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:55.330059 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329e893-772b-4d80-a9a4-b84cad943f17 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:55.359217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b7eb0-8ca6-4667-b455-e8258ce5ea50 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:55.380022 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cef2a2-87ca-45bb-82bc-5602861c2f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:55.381555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef2a2-87ca-45bb-82bc-5602861c2f26 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:55.382064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef2a2-87ca-45bb-82bc-5602861c2f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:55.382672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef2a2-87ca-45bb-82bc-5602861c2f26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:55.385575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:55.394687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cef2a2-87ca-45bb-82bc-5602861c2f26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:55.396600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cef2a2-87ca-45bb-82bc-5602861c2f26 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:55.403119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcca2b-9a6b-4c20-8565-5a385d22650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:55.403736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcca2b-9a6b-4c20-8565-5a385d22650a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:55.404032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcca2b-9a6b-4c20-8565-5a385d22650a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:55.404718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcca2b-9a6b-4c20-8565-5a385d22650a None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:55.405451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcca2b-9a6b-4c20-8565-5a385d22650a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:39:55.405759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcca2b-9a6b-4c20-8565-5a385d22650a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:55.407845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcca2b-9a6b-4c20-8565-5a385d22650a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b3cef2a2-87ca-45bb-82bc-5602861c2f26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:39:55.415432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:39:55.416400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:39:55.417458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:39:55.417458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:39:55.417458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:39:55.418927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 17/29] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:39:55.420005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cef2a2-87ca-45bb-82bc-5602861c2f26 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:55.427791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 13/30] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:39:55.429116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6160abd-5fea-4665-bf02-e45627a0ee23 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:55.429408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6160abd-5fea-4665-bf02-e45627a0ee23 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:55.429684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6160abd-5fea-4665-bf02-e45627a0ee23 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:55.431546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 14/31] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:39:55.432274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6160abd-5fea-4665-bf02-e45627a0ee23 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:55.481688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:55.538120 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cef2a2-87ca-45bb-82bc-5602861c2f26 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:55.548683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6160abd-5fea-4665-bf02-e45627a0ee23 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:55.574770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd243ab-d488-449a-bad9-56a329ecec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:55.575522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd243ab-d488-449a-bad9-56a329ecec36 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:55.575919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd243ab-d488-449a-bad9-56a329ecec36 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:55.576294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd243ab-d488-449a-bad9-56a329ecec36 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:55.577093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd243ab-d488-449a-bad9-56a329ecec36 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:55.578585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd243ab-d488-449a-bad9-56a329ecec36 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:55.588858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9da4e1-c080-4092-a415-deb9e63d656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:55.589510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9da4e1-c080-4092-a415-deb9e63d656c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:55.589809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9da4e1-c080-4092-a415-deb9e63d656c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:55.590089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9da4e1-c080-4092-a415-deb9e63d656c None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:55.590738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9da4e1-c080-4092-a415-deb9e63d656c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:39:55.591039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9da4e1-c080-4092-a415-deb9e63d656c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:55.592421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9da4e1-c080-4092-a415-deb9e63d656c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:55.604954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 15/32] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:39:55.605954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017576-4d8e-4b65-9978-ae699b3e6ed8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:55.606489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017576-4d8e-4b65-9978-ae699b3e6ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:55.606489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a017576-4d8e-4b65-9978-ae699b3e6ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:55.626759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5a017576-4d8e-4b65-9978-ae699b3e6ed8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:55.628004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5a017576-4d8e-4b65-9978-ae699b3e6ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:55.707567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 16/33] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:39:55.708742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017576-4d8e-4b65-9978-ae699b3e6ed8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:55.754042 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:55.790390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 18/34] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Aug 11 09:39:55 2022] POST /identity/v3/roles => generated 225 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:39:55.792222 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd243ab-d488-449a-bad9-56a329ecec36 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:55.835096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a017576-4d8e-4b65-9978-ae699b3e6ed8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:55.852526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:55.874489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596a49e-9fbf-445d-90d0-db3a6ee63ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:55.875149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596a49e-9fbf-445d-90d0-db3a6ee63ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:55.875501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596a49e-9fbf-445d-90d0-db3a6ee63ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:55.875802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596a49e-9fbf-445d-90d0-db3a6ee63ec8 None admin] PATH_INFO: `/v3/users/7936a8595af344cb984e181520011d53` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:55.877753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596a49e-9fbf-445d-90d0-db3a6ee63ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=7936a8595af344cb984e181520011d53)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:55.878885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2596a49e-9fbf-445d-90d0-db3a6ee63ec8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:55.880501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 17/35] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:39:55 2022] GET /identity/v3/users/7936a8595af344cb984e181520011d53 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:39:55.881388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596a49e-9fbf-445d-90d0-db3a6ee63ec8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:55.919477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd243ab-d488-449a-bad9-56a329ecec36 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:55.959473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f694a6-3bf2-47fb-a915-5e1930012200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:55.960518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f694a6-3bf2-47fb-a915-5e1930012200 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:55.961026 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f694a6-3bf2-47fb-a915-5e1930012200 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:55.962055 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f694a6-3bf2-47fb-a915-5e1930012200 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:55.962899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f694a6-3bf2-47fb-a915-5e1930012200 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:55.964685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f694a6-3bf2-47fb-a915-5e1930012200 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:55.989366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:56.000253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596a49e-9fbf-445d-90d0-db3a6ee63ec8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:56.043199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cda9f-14a1-4d24-8564-8b3658f0509c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:56.044219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cda9f-14a1-4d24-8564-8b3658f0509c None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:56.044735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cda9f-14a1-4d24-8564-8b3658f0509c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:56.045236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cda9f-14a1-4d24-8564-8b3658f0509c None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:56.046130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cda9f-14a1-4d24-8564-8b3658f0509c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:56.048073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cda9f-14a1-4d24-8564-8b3658f0509c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:56.087669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 19/36] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:39:55 2022] POST /identity/v3/projects => generated 301 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:39:56.088512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f694a6-3bf2-47fb-a915-5e1930012200 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:56.226543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:56.231390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f694a6-3bf2-47fb-a915-5e1930012200 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:56.270799 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c48bae-08c9-4caa-8e6d-31b0e06602b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:56.271505 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c48bae-08c9-4caa-8e6d-31b0e06602b4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:56.271921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c48bae-08c9-4caa-8e6d-31b0e06602b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:56.272108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c48bae-08c9-4caa-8e6d-31b0e06602b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:56.273330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c48bae-08c9-4caa-8e6d-31b0e06602b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:56.275067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c48bae-08c9-4caa-8e6d-31b0e06602b4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:56.276817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 20/37] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:39:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:39:56.279957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3926567-7d34-4fb0-9f1f-6d2b893c0607 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:56.279957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3926567-7d34-4fb0-9f1f-6d2b893c0607 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:56.279957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3926567-7d34-4fb0-9f1f-6d2b893c0607 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:56.280566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 18/38] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Aug 11 09:39:55 2022] POST /identity/v3/roles => generated 219 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:39:56.283622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 21/39] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:39:56.291263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cda9f-14a1-4d24-8564-8b3658f0509c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:56.292070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0a25d-69d7-475d-aabc-3e53cc78a33a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:56.292394 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0a25d-69d7-475d-aabc-3e53cc78a33a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:56.292773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0a25d-69d7-475d-aabc-3e53cc78a33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:56.315749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-bab0a25d-69d7-475d-aabc-3e53cc78a33a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:56.317985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bab0a25d-69d7-475d-aabc-3e53cc78a33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:56.396307 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 22/40] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:39:56.397914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6cbd2-1cd3-4343-95e8-7f5e17e09c31 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:56.398339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6cbd2-1cd3-4343-95e8-7f5e17e09c31 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:56.399440 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6cbd2-1cd3-4343-95e8-7f5e17e09c31 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:56.400913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 23/41] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:39:56.410616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be8790-3188-49e1-ad20-136f585d61a0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:56.410949 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be8790-3188-49e1-ad20-136f585d61a0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:56.411229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be8790-3188-49e1-ad20-136f585d61a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:56.434963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-96be8790-3188-49e1-ad20-136f585d61a0 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:56.436435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-96be8790-3188-49e1-ad20-136f585d61a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:56.450412 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cda9f-14a1-4d24-8564-8b3658f0509c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:56.491126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba0a8f-7264-4e30-a48b-b231cd1db882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:56.491868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba0a8f-7264-4e30-a48b-b231cd1db882 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:56.492156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba0a8f-7264-4e30-a48b-b231cd1db882 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:56.492763 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba0a8f-7264-4e30-a48b-b231cd1db882 None admin] PATH_INFO: `/v3/domains/default/users/7936a8595af344cb984e181520011d53/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:56.495157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba0a8f-7264-4e30-a48b-b231cd1db882 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7936a8595af344cb984e181520011d53, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:56.496741 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba0a8f-7264-4e30-a48b-b231cd1db882 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:56.514304 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 24/42] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:39:56.515165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be8790-3188-49e1-ad20-136f585d61a0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:56.543954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:56.602175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 19/43] 10.222.0.17 () {58 vars in 1361 bytes} [Thu Aug 11 09:39:56 2022] PUT /identity/v3/domains/default/users/7936a8595af344cb984e181520011d53/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:39:56.603012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba0a8f-7264-4e30-a48b-b231cd1db882 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:56.615513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be8790-3188-49e1-ad20-136f585d61a0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:56.625228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:56.656929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a40165-2c4c-41de-8094-2ff6738669fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:56.657598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a40165-2c4c-41de-8094-2ff6738669fc None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:56.657880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a40165-2c4c-41de-8094-2ff6738669fc None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:56.658154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a40165-2c4c-41de-8094-2ff6738669fc None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:56.658974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a40165-2c4c-41de-8094-2ff6738669fc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:56.660431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a40165-2c4c-41de-8094-2ff6738669fc None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:56.711981 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 25/44] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Aug 11 09:39:56 2022] POST /identity/v3/roles => generated 223 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:39:56.722293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba0a8f-7264-4e30-a48b-b231cd1db882 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:56.768547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadea8f-21d2-4644-b475-6cd3c49d8c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:56.769192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadea8f-21d2-4644-b475-6cd3c49d8c23 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:56.769454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadea8f-21d2-4644-b475-6cd3c49d8c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:56.769735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadea8f-21d2-4644-b475-6cd3c49d8c23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:56.770955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadea8f-21d2-4644-b475-6cd3c49d8c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:56.772420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadea8f-21d2-4644-b475-6cd3c49d8c23 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:56.774212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 20/45] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:39:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:39:56.780758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a40165-2c4c-41de-8094-2ff6738669fc None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:56.878753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:56.894494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a40165-2c4c-41de-8094-2ff6738669fc None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:56.931175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9640a-9a91-4b47-b8c8-0b6007e06247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:56.931949 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9640a-9a91-4b47-b8c8-0b6007e06247 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:56.932395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9640a-9a91-4b47-b8c8-0b6007e06247 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:56.933037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9640a-9a91-4b47-b8c8-0b6007e06247 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:56.934058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9640a-9a91-4b47-b8c8-0b6007e06247 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:56.936065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9640a-9a91-4b47-b8c8-0b6007e06247 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:57.002779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 26/46] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:39:56 2022] POST /identity/v3/projects => generated 312 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:39:58.637098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e7ecd-6d0c-405c-af57-5a06728076af None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:58.637605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e7ecd-6d0c-405c-af57-5a06728076af None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:58.637687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e7ecd-6d0c-405c-af57-5a06728076af None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:58.638993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 21/47] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:39:58.644234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce907-e168-449c-af76-1d99aa99b2dd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:58.644667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce907-e168-449c-af76-1d99aa99b2dd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:58.644803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ce907-e168-449c-af76-1d99aa99b2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:58.663197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2c7ce907-e168-449c-af76-1d99aa99b2dd None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:58.665145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2c7ce907-e168-449c-af76-1d99aa99b2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:58.732253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 27/48] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:39:58.755945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e7ecd-6d0c-405c-af57-5a06728076af None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:58.840776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:58.858190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e7ecd-6d0c-405c-af57-5a06728076af None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:58.891931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f693820-727f-48f3-98f2-87caead2c94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:58.892500 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693820-727f-48f3-98f2-87caead2c94b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:58.892786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693820-727f-48f3-98f2-87caead2c94b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:58.893087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f693820-727f-48f3-98f2-87caead2c94b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:58.903705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f693820-727f-48f3-98f2-87caead2c94b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-7f693820-727f-48f3-98f2-87caead2c94b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:39:58.908518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:39:58.909433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:39:58.910152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 22/49] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:39:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:39:58.916379 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ce907-e168-449c-af76-1d99aa99b2dd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:58.981512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:58.993451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d854c64-ef14-47eb-a67b-2f82f0e474d5 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:58.993701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d854c64-ef14-47eb-a67b-2f82f0e474d5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:58.994069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d854c64-ef14-47eb-a67b-2f82f0e474d5 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:58.995419 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 23/50] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:39:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:39:59.000236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a776c3-8567-4a77-8545-9150d69461d5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:59.000577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a776c3-8567-4a77-8545-9150d69461d5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:59.000852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a776c3-8567-4a77-8545-9150d69461d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:59.014308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ce907-e168-449c-af76-1d99aa99b2dd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:59.018851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-33a776c3-8567-4a77-8545-9150d69461d5 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:39:59.020396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-33a776c3-8567-4a77-8545-9150d69461d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:39:59.048619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9bdb9-8179-48ce-93cb-ef7628e4c843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:59.049359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9bdb9-8179-48ce-93cb-ef7628e4c843 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:59.049645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9bdb9-8179-48ce-93cb-ef7628e4c843 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:59.049922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9bdb9-8179-48ce-93cb-ef7628e4c843 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:59.050620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9bdb9-8179-48ce-93cb-ef7628e4c843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:39:59.050926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9bdb9-8179-48ce-93cb-ef7628e4c843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:59.052297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9bdb9-8179-48ce-93cb-ef7628e4c843 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:59.060345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 28/51] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:39:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:39:59.065873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9bdb9-8179-48ce-93cb-ef7628e4c843 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:59.089720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 24/52] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:39:59.111951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a776c3-8567-4a77-8545-9150d69461d5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:59.166437 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9bdb9-8179-48ce-93cb-ef7628e4c843 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:59.184644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:59.200830 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:59.206210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95aefd-c402-4e13-a189-a63326b119e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:59.206855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95aefd-c402-4e13-a189-a63326b119e2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:59.207132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95aefd-c402-4e13-a189-a63326b119e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:59.207409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95aefd-c402-4e13-a189-a63326b119e2 None admin] PATH_INFO: `/v3/users/7936a8595af344cb984e181520011d53` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:59.208559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95aefd-c402-4e13-a189-a63326b119e2 None admin] RBAC: Authorizing `identity:get_user(user_id=7936a8595af344cb984e181520011d53)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:59.209600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a776c3-8567-4a77-8545-9150d69461d5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:59.210033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95aefd-c402-4e13-a189-a63326b119e2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:59.212490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 29/53] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:39:59 2022] GET /identity/v3/users/7936a8595af344cb984e181520011d53 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:39:59.217693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95aefd-c402-4e13-a189-a63326b119e2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:59.245117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b837d-fa75-46b9-b13c-a701f55efd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:59.245756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b837d-fa75-46b9-b13c-a701f55efd83 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:59.246037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b837d-fa75-46b9-b13c-a701f55efd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:59.246311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b837d-fa75-46b9-b13c-a701f55efd83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:59.247427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b837d-fa75-46b9-b13c-a701f55efd83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:59.248867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b837d-fa75-46b9-b13c-a701f55efd83 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:59.250463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 25/54] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:39:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:39:59.255959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b837d-fa75-46b9-b13c-a701f55efd83 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:59.322094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95aefd-c402-4e13-a189-a63326b119e2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:59.354836 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b837d-fa75-46b9-b13c-a701f55efd83 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:59.355013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad6cf9-f563-48bb-8a99-271dbb510a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:59.355640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad6cf9-f563-48bb-8a99-271dbb510a90 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:59.355923 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad6cf9-f563-48bb-8a99-271dbb510a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:59.356201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad6cf9-f563-48bb-8a99-271dbb510a90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:59.357376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad6cf9-f563-48bb-8a99-271dbb510a90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:59.358819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad6cf9-f563-48bb-8a99-271dbb510a90 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:59.360388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 30/55] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:39:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:39:59.366079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad6cf9-f563-48bb-8a99-271dbb510a90 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:39:59.389128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d1623-5f79-49ed-b40e-e5c764b181bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:59.389747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d1623-5f79-49ed-b40e-e5c764b181bb None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:59.390031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d1623-5f79-49ed-b40e-e5c764b181bb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:59.390304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d1623-5f79-49ed-b40e-e5c764b181bb None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:59.390951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d1623-5f79-49ed-b40e-e5c764b181bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:59.392381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d1623-5f79-49ed-b40e-e5c764b181bb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:59.407907 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:59.434210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:39:59.464472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad6cf9-f563-48bb-8a99-271dbb510a90 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:39:59.489846 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 26/56] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Aug 11 09:39:59 2022] POST /identity/v3/projects => generated 298 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:39:59.496716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5041b-cb0a-40de-8e5d-a0b7a95790e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:39:59.497382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5041b-cb0a-40de-8e5d-a0b7a95790e3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:39:59.497713 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5041b-cb0a-40de-8e5d-a0b7a95790e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:39:59.497986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5041b-cb0a-40de-8e5d-a0b7a95790e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:39:59.498810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5041b-cb0a-40de-8e5d-a0b7a95790e3 None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.domain.id=default, user.id=7936a8595af344cb984e181520011d53) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:39:59.499066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5041b-cb0a-40de-8e5d-a0b7a95790e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:39:59.500971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5041b-cb0a-40de-8e5d-a0b7a95790e3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:39:59.521054 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 31/57] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Aug 11 09:39:59 2022] GET /identity/v3/role_assignments?user.id=7936a8595af344cb984e181520011d53&scope.domain.id=default&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:01.494096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980b893-bbc2-4456-bb17-6ce0e81a5655 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:01.494583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980b893-bbc2-4456-bb17-6ce0e81a5655 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:01.494583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980b893-bbc2-4456-bb17-6ce0e81a5655 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:01.495899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 27/58] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:01.501056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d4881-7a4f-454c-9bf9-c1cef78852e9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:01.501462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d4881-7a4f-454c-9bf9-c1cef78852e9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:01.501610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d4881-7a4f-454c-9bf9-c1cef78852e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:01.519713 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-250d4881-7a4f-454c-9bf9-c1cef78852e9 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:01.520916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-250d4881-7a4f-454c-9bf9-c1cef78852e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:01.585638 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 32/59] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:01.612427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980b893-bbc2-4456-bb17-6ce0e81a5655 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:01.713458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980b893-bbc2-4456-bb17-6ce0e81a5655 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:01.746068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a31be8-9d10-4da1-b798-978c207e37ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:01.746633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a31be8-9d10-4da1-b798-978c207e37ba None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:01.746919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a31be8-9d10-4da1-b798-978c207e37ba None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:01.747194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a31be8-9d10-4da1-b798-978c207e37ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:01.748313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a31be8-9d10-4da1-b798-978c207e37ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:01.749767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a31be8-9d10-4da1-b798-978c207e37ba None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:01.751317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 28/60] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:01.756861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d4881-7a4f-454c-9bf9-c1cef78852e9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:01.813714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:01.851658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d4881-7a4f-454c-9bf9-c1cef78852e9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:01.884753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eeea64-e7b0-4fc9-9f1e-25063dd786b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:01.885368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eeea64-e7b0-4fc9-9f1e-25063dd786b0 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:01.885648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eeea64-e7b0-4fc9-9f1e-25063dd786b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:01.885920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eeea64-e7b0-4fc9-9f1e-25063dd786b0 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:01.886568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eeea64-e7b0-4fc9-9f1e-25063dd786b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:01.887986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eeea64-e7b0-4fc9-9f1e-25063dd786b0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:01.997086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 33/61] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:40:01 2022] POST /identity/v3/users => generated 275 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:40:05.135447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089e3fd-1f03-48de-a0a8-b7c2b53a70cc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.135955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089e3fd-1f03-48de-a0a8-b7c2b53a70cc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.136066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089e3fd-1f03-48de-a0a8-b7c2b53a70cc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.137502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 29/62] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:05.142627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fb60c-0dc6-4f0b-b97e-24715056d6d6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.143020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fb60c-0dc6-4f0b-b97e-24715056d6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.143182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fb60c-0dc6-4f0b-b97e-24715056d6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.170002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c41fb60c-0dc6-4f0b-b97e-24715056d6d6 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:05.171246 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c41fb60c-0dc6-4f0b-b97e-24715056d6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:05.279464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 34/63] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:05.310514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089e3fd-1f03-48de-a0a8-b7c2b53a70cc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:05.324635 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6035b66-58c7-4999-8628-f4fc61b1e4c1 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.324941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6035b66-58c7-4999-8628-f4fc61b1e4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.325222 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6035b66-58c7-4999-8628-f4fc61b1e4c1 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.326698 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 35/64] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:05.333981 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef8c52-3389-4cb3-a6ea-09c0fe4ec6eb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.334264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef8c52-3389-4cb3-a6ea-09c0fe4ec6eb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.334543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef8c52-3389-4cb3-a6ea-09c0fe4ec6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.365414 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-afef8c52-3389-4cb3-a6ea-09c0fe4ec6eb None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:05.366633 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-afef8c52-3389-4cb3-a6ea-09c0fe4ec6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:05.417504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:05.449100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:05.460623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089e3fd-1f03-48de-a0a8-b7c2b53a70cc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:05.480944 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 36/65] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:05.482532 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74b7e5-4010-49ac-bad9-7b8ac57c2167 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.483018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74b7e5-4010-49ac-bad9-7b8ac57c2167 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.483488 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74b7e5-4010-49ac-bad9-7b8ac57c2167 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.484917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 37/66] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:05.501999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5c769-ea14-4c42-87ff-f50fc2d9b547 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.502579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5c769-ea14-4c42-87ff-f50fc2d9b547 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.503051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5c769-ea14-4c42-87ff-f50fc2d9b547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.515241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069076f-00d0-468e-8755-f90bc1b68f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:05.516373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069076f-00d0-468e-8755-f90bc1b68f91 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.516986 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069076f-00d0-468e-8755-f90bc1b68f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.517908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069076f-00d0-468e-8755-f90bc1b68f91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.526578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-34e5c769-ea14-4c42-87ff-f50fc2d9b547 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:05.528337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-34e5c769-ea14-4c42-87ff-f50fc2d9b547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:05.529962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069076f-00d0-468e-8755-f90bc1b68f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-c069076f-00d0-468e-8755-f90bc1b68f91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:05.534876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:05.535747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:05.536490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 30/67] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:05.537151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069076f-00d0-468e-8755-f90bc1b68f91 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:05.624111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 38/68] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:05.624749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5c769-ea14-4c42-87ff-f50fc2d9b547 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:05.649674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069076f-00d0-468e-8755-f90bc1b68f91 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:05.683687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:05.697693 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de55c94-67d8-40af-9d8e-9205aa6b6615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:05.698393 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de55c94-67d8-40af-9d8e-9205aa6b6615 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.698676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de55c94-67d8-40af-9d8e-9205aa6b6615 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.699064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de55c94-67d8-40af-9d8e-9205aa6b6615 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.711689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de55c94-67d8-40af-9d8e-9205aa6b6615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4de55c94-67d8-40af-9d8e-9205aa6b6615 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:05.716721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:05.717575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:05.718304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:05.723244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 31/69] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:40:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:05.724090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de55c94-67d8-40af-9d8e-9205aa6b6615 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:05.744817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5c769-ea14-4c42-87ff-f50fc2d9b547 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:05.753876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:05.783661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f62b-87bf-4d95-b72e-8d76c335bd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:05.784277 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f62b-87bf-4d95-b72e-8d76c335bd72 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.784559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f62b-87bf-4d95-b72e-8d76c335bd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.784831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28f62b-87bf-4d95-b72e-8d76c335bd72 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.785530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28f62b-87bf-4d95-b72e-8d76c335bd72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:05.785831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28f62b-87bf-4d95-b72e-8d76c335bd72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:05.787210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28f62b-87bf-4d95-b72e-8d76c335bd72 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:05.795894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 39/70] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:05.796697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f62b-87bf-4d95-b72e-8d76c335bd72 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:05.838646 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de55c94-67d8-40af-9d8e-9205aa6b6615 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:05.872756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313db22-b64d-4981-9426-6e727e1f58de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:05.873417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313db22-b64d-4981-9426-6e727e1f58de None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.873701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313db22-b64d-4981-9426-6e727e1f58de None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.873978 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313db22-b64d-4981-9426-6e727e1f58de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.884490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313db22-b64d-4981-9426-6e727e1f58de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2313db22-b64d-4981-9426-6e727e1f58de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:05.889207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:05.890063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:05.891450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 32/71] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:05.892236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e0a81-a7ea-473f-a1f0-8f1b573bac65 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.892236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e0a81-a7ea-473f-a1f0-8f1b573bac65 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.892236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e0a81-a7ea-473f-a1f0-8f1b573bac65 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.893506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 33/72] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:05.894510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e970a50-5c50-4021-8fe1-eb2696df8488 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.894796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e970a50-5c50-4021-8fe1-eb2696df8488 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.895072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e970a50-5c50-4021-8fe1-eb2696df8488 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.896246 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 34/73] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:05.896938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e970a50-5c50-4021-8fe1-eb2696df8488 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:05.902706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28f62b-87bf-4d95-b72e-8d76c335bd72 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:05.935690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea277ba4-fe7b-406a-92cf-065e70f1e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:05.936221 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea277ba4-fe7b-406a-92cf-065e70f1e39b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:05.936502 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea277ba4-fe7b-406a-92cf-065e70f1e39b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:05.936772 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea277ba4-fe7b-406a-92cf-065e70f1e39b None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:05.937464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea277ba4-fe7b-406a-92cf-065e70f1e39b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:05.937765 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea277ba4-fe7b-406a-92cf-065e70f1e39b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:05.939146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea277ba4-fe7b-406a-92cf-065e70f1e39b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:05.941795 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:05.948695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 40/74] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:05.949673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea277ba4-fe7b-406a-92cf-065e70f1e39b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:05.980657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:06.000252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e970a50-5c50-4021-8fe1-eb2696df8488 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:06.033257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a98ad2-8cf5-41ca-80a4-ed05734ac9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:06.033895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a98ad2-8cf5-41ca-80a4-ed05734ac9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:06.034180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a98ad2-8cf5-41ca-80a4-ed05734ac9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:06.034458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a98ad2-8cf5-41ca-80a4-ed05734ac9c0 None admin] PATH_INFO: `/v3/users/0b773d1df61d4a3b933926aeff31873c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:06.049859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a98ad2-8cf5-41ca-80a4-ed05734ac9c0 None admin] RBAC: Authorizing `identity:get_user(user_id=0b773d1df61d4a3b933926aeff31873c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:06.050314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea277ba4-fe7b-406a-92cf-065e70f1e39b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:06.051310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a98ad2-8cf5-41ca-80a4-ed05734ac9c0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:06.052908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 35/75] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:05 2022] GET /identity/v3/users/0b773d1df61d4a3b933926aeff31873c => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:06.054067 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71036a66-58ba-403d-b43d-d547b130b992 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:06.054350 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71036a66-58ba-403d-b43d-d547b130b992 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:06.055257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71036a66-58ba-403d-b43d-d547b130b992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:06.073415 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-71036a66-58ba-403d-b43d-d547b130b992 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:06.074618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-71036a66-58ba-403d-b43d-d547b130b992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:06.088530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3cbd0-af1c-425e-bec4-2939915f7e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:06.089173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3cbd0-af1c-425e-bec4-2939915f7e95 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:06.089466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3cbd0-af1c-425e-bec4-2939915f7e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:06.089739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3cbd0-af1c-425e-bec4-2939915f7e95 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:06.090413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3cbd0-af1c-425e-bec4-2939915f7e95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:06.090721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3cbd0-af1c-425e-bec4-2939915f7e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:06.092090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3cbd0-af1c-425e-bec4-2939915f7e95 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:06.099951 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 41/76] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:06.101827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c88c-4fe6-406f-bf75-d93515bd917b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:06.102184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c88c-4fe6-406f-bf75-d93515bd917b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:06.102557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82c88c-4fe6-406f-bf75-d93515bd917b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:06.120056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-bf82c88c-4fe6-406f-bf75-d93515bd917b None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:06.121310 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bf82c88c-4fe6-406f-bf75-d93515bd917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:06.143140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 36/77] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:06.144064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71036a66-58ba-403d-b43d-d547b130b992 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:06.172864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:06.205452 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 42/78] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:06.206881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82c88c-4fe6-406f-bf75-d93515bd917b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:06.230058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:06.248193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71036a66-58ba-403d-b43d-d547b130b992 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:06.282352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51398f9c-cb3b-43f4-87e7-cb71eaab8428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:06.282988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51398f9c-cb3b-43f4-87e7-cb71eaab8428 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:06.283265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51398f9c-cb3b-43f4-87e7-cb71eaab8428 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:06.283540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51398f9c-cb3b-43f4-87e7-cb71eaab8428 None admin] PATH_INFO: `/v3/users/7936a8595af344cb984e181520011d53` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:06.284679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51398f9c-cb3b-43f4-87e7-cb71eaab8428 None admin] RBAC: Authorizing `identity:get_user(user_id=7936a8595af344cb984e181520011d53)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:06.286140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51398f9c-cb3b-43f4-87e7-cb71eaab8428 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:06.287693 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 37/79] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:06 2022] GET /identity/v3/users/7936a8595af344cb984e181520011d53 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:06.288453 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51398f9c-cb3b-43f4-87e7-cb71eaab8428 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:06.306856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82c88c-4fe6-406f-bf75-d93515bd917b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:06.340318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dc278-78c4-41d8-a675-a1600543bbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:06.340927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dc278-78c4-41d8-a675-a1600543bbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:06.341240 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dc278-78c4-41d8-a675-a1600543bbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:06.341520 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67dc278-78c4-41d8-a675-a1600543bbb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:06.384063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:06.387752 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51398f9c-cb3b-43f4-87e7-cb71eaab8428 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:06.406984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67dc278-78c4-41d8-a675-a1600543bbb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:06.408442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67dc278-78c4-41d8-a675-a1600543bbb6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:06.421299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1b014-38ca-41fa-925c-b7cb6ffe28a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:06.421934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1b014-38ca-41fa-925c-b7cb6ffe28a4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:06.422220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1b014-38ca-41fa-925c-b7cb6ffe28a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:06.422500 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1b014-38ca-41fa-925c-b7cb6ffe28a4 None admin] PATH_INFO: `/v3/users/0b773d1df61d4a3b933926aeff31873c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:06.423648 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1b014-38ca-41fa-925c-b7cb6ffe28a4 None admin] RBAC: Authorizing `identity:get_user(user_id=0b773d1df61d4a3b933926aeff31873c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:06.425106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1b014-38ca-41fa-925c-b7cb6ffe28a4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:06.426679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 38/80] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:06 2022] GET /identity/v3/users/0b773d1df61d4a3b933926aeff31873c => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:06.427440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1b014-38ca-41fa-925c-b7cb6ffe28a4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:06.435197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:06.516000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 43/81] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:06.516000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dc278-78c4-41d8-a675-a1600543bbb6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:06.528999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1b014-38ca-41fa-925c-b7cb6ffe28a4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:06.562715 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bb54f-32de-47c7-8447-44f53ed35b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:06.563342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bb54f-32de-47c7-8447-44f53ed35b92 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:06.563620 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bb54f-32de-47c7-8447-44f53ed35b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:06.563893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bb54f-32de-47c7-8447-44f53ed35b92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:06.565241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67bb54f-32de-47c7-8447-44f53ed35b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:06.566458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67bb54f-32de-47c7-8447-44f53ed35b92 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:06.568261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 39/82] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:06.569020 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bb54f-32de-47c7-8447-44f53ed35b92 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:06.603687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:06.612380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67dc278-78c4-41d8-a675-a1600543bbb6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:06.635060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:06.646466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130118c-5b3c-41ec-8c4d-691b3fd62fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:06.647198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130118c-5b3c-41ec-8c4d-691b3fd62fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:06.647477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130118c-5b3c-41ec-8c4d-691b3fd62fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:06.647757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130118c-5b3c-41ec-8c4d-691b3fd62fe7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:06.658656 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130118c-5b3c-41ec-8c4d-691b3fd62fe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b130118c-5b3c-41ec-8c4d-691b3fd62fe7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:06.664335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:06.665242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:06.665980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 44/83] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:06.666805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130118c-5b3c-41ec-8c4d-691b3fd62fe7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:06.674650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bb54f-32de-47c7-8447-44f53ed35b92 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:06.708840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094b31e-a16c-45e5-9dbc-e610ae332fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:06.709489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094b31e-a16c-45e5-9dbc-e610ae332fda None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:06.709774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094b31e-a16c-45e5-9dbc-e610ae332fda None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:06.710050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094b31e-a16c-45e5-9dbc-e610ae332fda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:06.766493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130118c-5b3c-41ec-8c4d-691b3fd62fe7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:06.776289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094b31e-a16c-45e5-9dbc-e610ae332fda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:06.777762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094b31e-a16c-45e5-9dbc-e610ae332fda None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:06.800217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a2284-4298-463c-b211-62b225940a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:06.800832 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a2284-4298-463c-b211-62b225940a4a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:06.801143 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a2284-4298-463c-b211-62b225940a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:06.801423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a2284-4298-463c-b211-62b225940a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:06.812160 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:06.850720 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:06.869762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a2284-4298-463c-b211-62b225940a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:06.871691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a2284-4298-463c-b211-62b225940a4a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:06.879017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 40/84] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:06.880099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094b31e-a16c-45e5-9dbc-e610ae332fda None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:06.970844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 45/85] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:06.971678 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a2284-4298-463c-b211-62b225940a4a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:06.978962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094b31e-a16c-45e5-9dbc-e610ae332fda None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:07.013136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbf146-424e-4e15-b380-167181094e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:07.013771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbf146-424e-4e15-b380-167181094e19 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:07.014058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbf146-424e-4e15-b380-167181094e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:07.014333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbf146-424e-4e15-b380-167181094e19 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:07.015471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbf146-424e-4e15-b380-167181094e19 None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:07.016967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbf146-424e-4e15-b380-167181094e19 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:07.018549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 41/86] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:06 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:07.019315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbf146-424e-4e15-b380-167181094e19 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:07.021783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:07.053784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:07.072662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a2284-4298-463c-b211-62b225940a4a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:07.106734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f8f27-1c25-4ffd-b1e8-7b510535b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:07.107342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f8f27-1c25-4ffd-b1e8-7b510535b342 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:07.107625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f8f27-1c25-4ffd-b1e8-7b510535b342 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:07.107896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f8f27-1c25-4ffd-b1e8-7b510535b342 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:07.108531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f8f27-1c25-4ffd-b1e8-7b510535b342 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:07.110417 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f8f27-1c25-4ffd-b1e8-7b510535b342 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:07.120179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbf146-424e-4e15-b380-167181094e19 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:07.152981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b569d-6231-4717-b5ce-98f0e2b8f0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:07.153613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b569d-6231-4717-b5ce-98f0e2b8f0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:07.153893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b569d-6231-4717-b5ce-98f0e2b8f0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:07.154170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b569d-6231-4717-b5ce-98f0e2b8f0e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:07.154825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b569d-6231-4717-b5ce-98f0e2b8f0e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:07.155132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b569d-6231-4717-b5ce-98f0e2b8f0e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:07.156492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b569d-6231-4717-b5ce-98f0e2b8f0e3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:07.166183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 42/87] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:07.166968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b569d-6231-4717-b5ce-98f0e2b8f0e3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:07.214051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 46/88] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:40:06 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:40:07.214899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f8f27-1c25-4ffd-b1e8-7b510535b342 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:07.228547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:07.272513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b569d-6231-4717-b5ce-98f0e2b8f0e3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:07.314931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcca062-9466-4c73-8a19-8830f8d2e732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:07.315588 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcca062-9466-4c73-8a19-8830f8d2e732 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:07.315874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcca062-9466-4c73-8a19-8830f8d2e732 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:07.316145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcca062-9466-4c73-8a19-8830f8d2e732 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:07.317705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcca062-9466-4c73-8a19-8830f8d2e732 None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:07.319698 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcca062-9466-4c73-8a19-8830f8d2e732 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:07.326424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 43/89] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:07 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:07.326424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcca062-9466-4c73-8a19-8830f8d2e732 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:07.346300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f8f27-1c25-4ffd-b1e8-7b510535b342 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:07.387931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7adee-399e-42bd-9567-d02605616b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:07.388578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7adee-399e-42bd-9567-d02605616b9b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:07.388861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7adee-399e-42bd-9567-d02605616b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:07.389173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7adee-399e-42bd-9567-d02605616b9b None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:07.390362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7adee-399e-42bd-9567-d02605616b9b None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:07.391901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7adee-399e-42bd-9567-d02605616b9b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:07.393507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 47/90] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:07 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:07.394281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7adee-399e-42bd-9567-d02605616b9b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:07.424088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:07.436707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcca062-9466-4c73-8a19-8830f8d2e732 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:07.472340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:07.477518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d78e8-ee7b-40ff-a6c3-9b2f8af197b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:07.478140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d78e8-ee7b-40ff-a6c3-9b2f8af197b8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:07.478426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d78e8-ee7b-40ff-a6c3-9b2f8af197b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:07.478697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d78e8-ee7b-40ff-a6c3-9b2f8af197b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:07.479446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d78e8-ee7b-40ff-a6c3-9b2f8af197b8 None admin] RBAC: Adding query filter params (role.id=bda5c75a326e4f5d9503960830898895, scope.project.id=393ec3a506734c8b8c02823cb21848e2, user.id=0b773d1df61d4a3b933926aeff31873c) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:07.479756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d78e8-ee7b-40ff-a6c3-9b2f8af197b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:07.481243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d78e8-ee7b-40ff-a6c3-9b2f8af197b8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:07.489052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 44/91] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:07 2022] GET /identity/v3/role_assignments?user.id=0b773d1df61d4a3b933926aeff31873c&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=bda5c75a326e4f5d9503960830898895 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:07.489868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d78e8-ee7b-40ff-a6c3-9b2f8af197b8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:07.550695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7adee-399e-42bd-9567-d02605616b9b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:07.599968 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef6747-edf1-4a92-a376-8b3bbb9d9c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:07.600614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef6747-edf1-4a92-a376-8b3bbb9d9c1a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:07.600896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef6747-edf1-4a92-a376-8b3bbb9d9c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:07.602312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef6747-edf1-4a92-a376-8b3bbb9d9c1a None admin] PATH_INFO: `/v3/users/0b773d1df61d4a3b933926aeff31873c` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:07.603780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef6747-edf1-4a92-a376-8b3bbb9d9c1a None admin] RBAC: Authorizing `identity:get_user(user_id=0b773d1df61d4a3b933926aeff31873c)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:07.605284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef6747-edf1-4a92-a376-8b3bbb9d9c1a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:07.606945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 48/92] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:07 2022] GET /identity/v3/users/0b773d1df61d4a3b933926aeff31873c => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:07.607804 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef6747-edf1-4a92-a376-8b3bbb9d9c1a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:07.634062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d78e8-ee7b-40ff-a6c3-9b2f8af197b8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:07.685462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004989d2-fe7b-4cf0-a431-0f8a36b696de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:07.686089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004989d2-fe7b-4cf0-a431-0f8a36b696de None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:07.686373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004989d2-fe7b-4cf0-a431-0f8a36b696de None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:07.686656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004989d2-fe7b-4cf0-a431-0f8a36b696de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:07.687448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004989d2-fe7b-4cf0-a431-0f8a36b696de None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=393ec3a506734c8b8c02823cb21848e2, user.id=7936a8595af344cb984e181520011d53) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:07.687758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004989d2-fe7b-4cf0-a431-0f8a36b696de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:07.689976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004989d2-fe7b-4cf0-a431-0f8a36b696de None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:07.696943 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 45/93] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:07 2022] GET /identity/v3/role_assignments?user.id=7936a8595af344cb984e181520011d53&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:07.697937 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004989d2-fe7b-4cf0-a431-0f8a36b696de None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:07.713397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:07.760591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef6747-edf1-4a92-a376-8b3bbb9d9c1a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:07.811416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:07.833856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54050418-dbf6-4a19-88a5-35936841f296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:07.834532 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050418-dbf6-4a19-88a5-35936841f296 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:07.834796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050418-dbf6-4a19-88a5-35936841f296 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:07.835076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050418-dbf6-4a19-88a5-35936841f296 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:07.836025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54050418-dbf6-4a19-88a5-35936841f296 None admin] RBAC: Adding query filter params (role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596, scope.project.id=393ec3a506734c8b8c02823cb21848e2, user.id=0b773d1df61d4a3b933926aeff31873c) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:07.836506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54050418-dbf6-4a19-88a5-35936841f296 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:07.838389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54050418-dbf6-4a19-88a5-35936841f296 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:07.858835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 49/94] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:07 2022] GET /identity/v3/role_assignments?user.id=0b773d1df61d4a3b933926aeff31873c&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:07.884739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004989d2-fe7b-4cf0-a431-0f8a36b696de None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:07.945193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474883c-6658-49e6-9a8c-66de2f07142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:07.946747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474883c-6658-49e6-9a8c-66de2f07142f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:07.947234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474883c-6658-49e6-9a8c-66de2f07142f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:07.947707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474883c-6658-49e6-9a8c-66de2f07142f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:08.030883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474883c-6658-49e6-9a8c-66de2f07142f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:08.032723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474883c-6658-49e6-9a8c-66de2f07142f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:08.108826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:08.171484 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 46/95] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:08.177354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54050418-dbf6-4a19-88a5-35936841f296 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:08.329857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54050418-dbf6-4a19-88a5-35936841f296 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:08.370105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876de61-bd6d-4615-8157-607e0fb7d9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:08.370675 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876de61-bd6d-4615-8157-607e0fb7d9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:08.370962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876de61-bd6d-4615-8157-607e0fb7d9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:08.371236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876de61-bd6d-4615-8157-607e0fb7d9d4 None admin] PATH_INFO: `/v3/projects/afb396faf072409096904eea3b792505` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:08.372413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876de61-bd6d-4615-8157-607e0fb7d9d4 None admin] RBAC: Authorizing `identity:get_project(project_id=afb396faf072409096904eea3b792505)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:08.373880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876de61-bd6d-4615-8157-607e0fb7d9d4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:08.375519 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 50/96] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:08 2022] GET /identity/v3/projects/afb396faf072409096904eea3b792505 => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:08.381772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474883c-6658-49e6-9a8c-66de2f07142f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:08.529626 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474883c-6658-49e6-9a8c-66de2f07142f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:08.579047 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39a055-bcc2-43de-8386-f1979af1235c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:08.580042 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39a055-bcc2-43de-8386-f1979af1235c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:08.580524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39a055-bcc2-43de-8386-f1979af1235c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:08.580998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39a055-bcc2-43de-8386-f1979af1235c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:08.581988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39a055-bcc2-43de-8386-f1979af1235c None admin] RBAC: Adding query filter params (role.id=bda5c75a326e4f5d9503960830898895, scope.project.id=afb396faf072409096904eea3b792505, user.id=0b773d1df61d4a3b933926aeff31873c) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:08.582517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39a055-bcc2-43de-8386-f1979af1235c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:08.584154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39a055-bcc2-43de-8386-f1979af1235c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:08.593219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 47/97] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:08 2022] GET /identity/v3/role_assignments?user.id=0b773d1df61d4a3b933926aeff31873c&scope.project.id=afb396faf072409096904eea3b792505&role.id=bda5c75a326e4f5d9503960830898895 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:10.240346 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83833de-967b-4c90-9f3d-832aa61f8eac None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:10.241391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83833de-967b-4c90-9f3d-832aa61f8eac None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:10.242252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83833de-967b-4c90-9f3d-832aa61f8eac None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:10.244157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 51/98] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:10.250104 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97476f94-b1b4-44fd-aa3a-f9cb4f9207a8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:10.251038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97476f94-b1b4-44fd-aa3a-f9cb4f9207a8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:10.251534 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97476f94-b1b4-44fd-aa3a-f9cb4f9207a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:10.278089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-97476f94-b1b4-44fd-aa3a-f9cb4f9207a8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:10.279639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-97476f94-b1b4-44fd-aa3a-f9cb4f9207a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:10.305070 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:10.363818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 48/99] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:10.397273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83833de-967b-4c90-9f3d-832aa61f8eac None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:10.407582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892f5ff-c48f-487b-b607-8b78afb0998a None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:10.407881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892f5ff-c48f-487b-b607-8b78afb0998a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:10.408168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892f5ff-c48f-487b-b607-8b78afb0998a None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:10.409551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 49/100] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:10.415590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23372b1e-8549-4ef2-b569-53ee4f0f2a89 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:10.419405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23372b1e-8549-4ef2-b569-53ee4f0f2a89 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:10.419405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23372b1e-8549-4ef2-b569-53ee4f0f2a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:10.441720 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:10.461442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-23372b1e-8549-4ef2-b569-53ee4f0f2a89 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:10.461442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-23372b1e-8549-4ef2-b569-53ee4f0f2a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:10.624121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83833de-967b-4c90-9f3d-832aa61f8eac None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:10.646892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 50/101] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:10.688337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00f918-0f5a-4122-b328-b9514cb03fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:10.688996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00f918-0f5a-4122-b328-b9514cb03fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:10.689433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00f918-0f5a-4122-b328-b9514cb03fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:10.689668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00f918-0f5a-4122-b328-b9514cb03fd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:10.690666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23372b1e-8549-4ef2-b569-53ee4f0f2a89 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:10.690841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00f918-0f5a-4122-b328-b9514cb03fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:10.692724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00f918-0f5a-4122-b328-b9514cb03fd5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:10.694601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 52/102] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:10.700368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00f918-0f5a-4122-b328-b9514cb03fd5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:10.833385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:10.837586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:10.860725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23372b1e-8549-4ef2-b569-53ee4f0f2a89 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:10.885098 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00f918-0f5a-4122-b328-b9514cb03fd5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:10.905603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385addb-746f-4d6a-b0f0-c1abb3522839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:10.906593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385addb-746f-4d6a-b0f0-c1abb3522839 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:10.906593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385addb-746f-4d6a-b0f0-c1abb3522839 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:10.907028 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385addb-746f-4d6a-b0f0-c1abb3522839 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:10.919191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385addb-746f-4d6a-b0f0-c1abb3522839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:10.922106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc09b-1cdd-4e7b-b963-5804e5da6edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:10.922914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc09b-1cdd-4e7b-b963-5804e5da6edd None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:10.923336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc09b-1cdd-4e7b-b963-5804e5da6edd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:10.923753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc09b-1cdd-4e7b-b963-5804e5da6edd None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4385addb-746f-4d6a-b0f0-c1abb3522839 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:10.924369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:10.925388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:10.926338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:10.927315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfc09b-1cdd-4e7b-b963-5804e5da6edd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:10.927672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 51/103] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:10.934673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26e759-0e1d-4782-acb4-9e6f5a5e88d3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:10.934673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26e759-0e1d-4782-acb4-9e6f5a5e88d3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:10.934673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26e759-0e1d-4782-acb4-9e6f5a5e88d3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:10.934673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 52/104] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:10.934673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ef6a9-c9de-4a36-9c70-29314d861fb2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:10.934673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ef6a9-c9de-4a36-9c70-29314d861fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:10.935081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ef6a9-c9de-4a36-9c70-29314d861fb2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:10.935379 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfc09b-1cdd-4e7b-b963-5804e5da6edd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:10.939462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 53/105] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:10.940530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb90ad-205a-481f-8490-fee508b67eeb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:10.940901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb90ad-205a-481f-8490-fee508b67eeb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:10.941400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb90ad-205a-481f-8490-fee508b67eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:10.961594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d1bb90ad-205a-481f-8490-fee508b67eeb None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:10.963134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d1bb90ad-205a-481f-8490-fee508b67eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:11.023458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 53/106] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:40:10 2022] POST /identity/v3/projects => generated 302 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:40:11.025557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc09b-1cdd-4e7b-b963-5804e5da6edd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:11.037041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 54/107] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:11.042378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fee0e-c6ee-4b04-81a9-28a2fba20964 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.042653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fee0e-c6ee-4b04-81a9-28a2fba20964 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.042891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fee0e-c6ee-4b04-81a9-28a2fba20964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.079849 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-856fee0e-c6ee-4b04-81a9-28a2fba20964 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:11.081124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-856fee0e-c6ee-4b04-81a9-28a2fba20964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:11.153887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc09b-1cdd-4e7b-b963-5804e5da6edd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:11.177916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 55/108] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:11.183281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fee0e-c6ee-4b04-81a9-28a2fba20964 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:11.191047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:11.191787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:11.207411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945260a0-50e7-4d8e-ab45-d237a462b548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:11.212357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945260a0-50e7-4d8e-ab45-d237a462b548 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.212357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945260a0-50e7-4d8e-ab45-d237a462b548 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.212357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945260a0-50e7-4d8e-ab45-d237a462b548 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.212357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945260a0-50e7-4d8e-ab45-d237a462b548 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:11.212357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945260a0-50e7-4d8e-ab45-d237a462b548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:11.213530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945260a0-50e7-4d8e-ab45-d237a462b548 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:11.225669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 54/109] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:11.232489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945260a0-50e7-4d8e-ab45-d237a462b548 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:11.303590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fee0e-c6ee-4b04-81a9-28a2fba20964 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:11.349370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63134c-65a6-4e84-ac24-e16f5002c2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:11.350021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63134c-65a6-4e84-ac24-e16f5002c2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.350306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63134c-65a6-4e84-ac24-e16f5002c2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.350585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63134c-65a6-4e84-ac24-e16f5002c2e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.351591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945260a0-50e7-4d8e-ab45-d237a462b548 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:11.370363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63134c-65a6-4e84-ac24-e16f5002c2e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0f63134c-65a6-4e84-ac24-e16f5002c2e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:11.377504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:11.378911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:11.379654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 56/110] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:40:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:11.380245 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63134c-65a6-4e84-ac24-e16f5002c2e2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:11.404994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c2bcc-5ab0-496f-a4cf-9a1840432222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:11.405776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c2bcc-5ab0-496f-a4cf-9a1840432222 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.406061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c2bcc-5ab0-496f-a4cf-9a1840432222 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.406338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c2bcc-5ab0-496f-a4cf-9a1840432222 None admin] PATH_INFO: `/v3/users/496733ccc8ef4cfea9edc82207092046` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.466050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:11.468322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c2bcc-5ab0-496f-a4cf-9a1840432222 None admin] RBAC: Authorizing `identity:get_user(user_id=496733ccc8ef4cfea9edc82207092046)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:11.470148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c2bcc-5ab0-496f-a4cf-9a1840432222 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:11.472824 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 55/111] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:11 2022] GET /identity/v3/users/496733ccc8ef4cfea9edc82207092046 => generated 289 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:11.473803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c2bcc-5ab0-496f-a4cf-9a1840432222 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:11.512765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63134c-65a6-4e84-ac24-e16f5002c2e2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:11.527418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:11.550904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c3552-4379-4bf6-989e-f27243a73ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:11.551539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c3552-4379-4bf6-989e-f27243a73ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.551824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c3552-4379-4bf6-989e-f27243a73ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.552101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c3552-4379-4bf6-989e-f27243a73ab0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.564161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c3552-4379-4bf6-989e-f27243a73ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-799c3552-4379-4bf6-989e-f27243a73ab0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:11.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:11.571036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 57/112] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:11.577787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2e544-3e5e-4635-a398-e82c1a3a37b4 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.578631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2e544-3e5e-4635-a398-e82c1a3a37b4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.578631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2e544-3e5e-4635-a398-e82c1a3a37b4 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.578631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 58/113] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:11.578631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2e544-3e5e-4635-a398-e82c1a3a37b4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:11.592701 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c2bcc-5ab0-496f-a4cf-9a1840432222 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:11.630042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b068c5f-ec7b-4768-9ed6-799bc94c9101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:11.630766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b068c5f-ec7b-4768-9ed6-799bc94c9101 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.631049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b068c5f-ec7b-4768-9ed6-799bc94c9101 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.631358 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b068c5f-ec7b-4768-9ed6-799bc94c9101 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.631994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b068c5f-ec7b-4768-9ed6-799bc94c9101 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:11.632321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b068c5f-ec7b-4768-9ed6-799bc94c9101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:11.634454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b068c5f-ec7b-4768-9ed6-799bc94c9101 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:11.648004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 56/114] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:11.649340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd2668-52ba-422b-bfe2-0dc555ac3eb1 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.649649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd2668-52ba-422b-bfe2-0dc555ac3eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.649926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd2668-52ba-422b-bfe2-0dc555ac3eb1 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.651301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 57/115] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:11.651858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd2668-52ba-422b-bfe2-0dc555ac3eb1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:11.689421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2e544-3e5e-4635-a398-e82c1a3a37b4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:11.715343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:11.726928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a93a4e-c9f7-4c28-b374-9ebf82448869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:11.727661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a93a4e-c9f7-4c28-b374-9ebf82448869 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.727967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a93a4e-c9f7-4c28-b374-9ebf82448869 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.728246 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a93a4e-c9f7-4c28-b374-9ebf82448869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.764431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd2668-52ba-422b-bfe2-0dc555ac3eb1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:11.775284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:11.801105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40a816-0ac4-4301-91de-d2565111ccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:11.801747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a816-0ac4-4301-91de-d2565111ccd9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.802040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a816-0ac4-4301-91de-d2565111ccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.802687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40a816-0ac4-4301-91de-d2565111ccd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.803751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40a816-0ac4-4301-91de-d2565111ccd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:11.804791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40a816-0ac4-4301-91de-d2565111ccd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:11.806178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a93a4e-c9f7-4c28-b374-9ebf82448869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:11.806814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc40a816-0ac4-4301-91de-d2565111ccd9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:11.807796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a93a4e-c9f7-4c28-b374-9ebf82448869 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:11.816347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 58/116] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:11.818125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22246367-5568-4a33-8fbf-8d6d7159c31c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.818407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22246367-5568-4a33-8fbf-8d6d7159c31c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.818682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22246367-5568-4a33-8fbf-8d6d7159c31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.838652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-22246367-5568-4a33-8fbf-8d6d7159c31c None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:11.840080 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-22246367-5568-4a33-8fbf-8d6d7159c31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:11.920451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 59/117] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:11.920637 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 59/118] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:11.920749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22246367-5568-4a33-8fbf-8d6d7159c31c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:11.920832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb348f-81fc-44fd-94a5-698fa74d6b6b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:11.920832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb348f-81fc-44fd-94a5-698fa74d6b6b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:11.920832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb348f-81fc-44fd-94a5-698fa74d6b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:11.934607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d9eb348f-81fc-44fd-94a5-698fa74d6b6b None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:11.935983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d9eb348f-81fc-44fd-94a5-698fa74d6b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:11.955767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:12.009140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 60/119] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:12.009900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb348f-81fc-44fd-94a5-698fa74d6b6b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:12.034039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:12.037725 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22246367-5568-4a33-8fbf-8d6d7159c31c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:12.079564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4296e-99cb-47a6-bb14-eaad2ca463a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:12.080362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4296e-99cb-47a6-bb14-eaad2ca463a5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:12.080961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4296e-99cb-47a6-bb14-eaad2ca463a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:12.081656 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4296e-99cb-47a6-bb14-eaad2ca463a5 None admin] PATH_INFO: `/v3/users/7936a8595af344cb984e181520011d53` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:12.083496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4296e-99cb-47a6-bb14-eaad2ca463a5 None admin] RBAC: Authorizing `identity:get_user(user_id=7936a8595af344cb984e181520011d53)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:12.085297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4296e-99cb-47a6-bb14-eaad2ca463a5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:12.091873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 60/120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:11 2022] GET /identity/v3/users/7936a8595af344cb984e181520011d53 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:12.091873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4296e-99cb-47a6-bb14-eaad2ca463a5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:12.129237 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb348f-81fc-44fd-94a5-698fa74d6b6b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:12.165790 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb276a4-b9e3-4ff1-a80d-bf52e0745d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:12.166437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb276a4-b9e3-4ff1-a80d-bf52e0745d90 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:12.166697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb276a4-b9e3-4ff1-a80d-bf52e0745d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:12.167425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb276a4-b9e3-4ff1-a80d-bf52e0745d90 None admin] PATH_INFO: `/v3/users/0b773d1df61d4a3b933926aeff31873c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:12.168558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb276a4-b9e3-4ff1-a80d-bf52e0745d90 None admin] RBAC: Authorizing `identity:get_user(user_id=0b773d1df61d4a3b933926aeff31873c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:12.170682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb276a4-b9e3-4ff1-a80d-bf52e0745d90 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:12.172471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 61/121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:12 2022] GET /identity/v3/users/0b773d1df61d4a3b933926aeff31873c => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:12.177532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb276a4-b9e3-4ff1-a80d-bf52e0745d90 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:12.196953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4296e-99cb-47a6-bb14-eaad2ca463a5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:12.208567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:12.233633 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb08b-65b4-469a-b4eb-9cf5d37ebc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:12.234277 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb08b-65b4-469a-b4eb-9cf5d37ebc9d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:12.234565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb08b-65b4-469a-b4eb-9cf5d37ebc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:12.234845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb08b-65b4-469a-b4eb-9cf5d37ebc9d None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:12.235993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eeb08b-65b4-469a-b4eb-9cf5d37ebc9d None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:12.237484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eeb08b-65b4-469a-b4eb-9cf5d37ebc9d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:12.239142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 61/122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:12 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:12.239993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb08b-65b4-469a-b4eb-9cf5d37ebc9d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:12.276735 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:12.297398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb276a4-b9e3-4ff1-a80d-bf52e0745d90 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:12.339803 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b83ef-3093-4923-9c10-ee191ee5d53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:12.340590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b83ef-3093-4923-9c10-ee191ee5d53f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:12.340884 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b83ef-3093-4923-9c10-ee191ee5d53f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:12.341194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b83ef-3093-4923-9c10-ee191ee5d53f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:12.346376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb08b-65b4-469a-b4eb-9cf5d37ebc9d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:12.352968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b83ef-3093-4923-9c10-ee191ee5d53f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4a7b83ef-3093-4923-9c10-ee191ee5d53f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:12.357902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:12.359396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 62/123] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:12.360142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b83ef-3093-4923-9c10-ee191ee5d53f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:12.388347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991af316-949a-4093-80ae-21f35f4212fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:12.389144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991af316-949a-4093-80ae-21f35f4212fb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:12.389493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991af316-949a-4093-80ae-21f35f4212fb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:12.389775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991af316-949a-4093-80ae-21f35f4212fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:12.401391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991af316-949a-4093-80ae-21f35f4212fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-991af316-949a-4093-80ae-21f35f4212fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:12.406391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:12.407568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 62/124] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:12.408481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991af316-949a-4093-80ae-21f35f4212fb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:12.457298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:12.479928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b83ef-3093-4923-9c10-ee191ee5d53f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:12.517001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366429c0-31e0-481c-af1c-0929e5a0ce5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:12.517787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366429c0-31e0-481c-af1c-0929e5a0ce5b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:12.518075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366429c0-31e0-481c-af1c-0929e5a0ce5b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:12.518360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366429c0-31e0-481c-af1c-0929e5a0ce5b None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:12.519533 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366429c0-31e0-481c-af1c-0929e5a0ce5b None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:12.521040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366429c0-31e0-481c-af1c-0929e5a0ce5b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:12.527684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 63/125] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:12 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:12.527684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366429c0-31e0-481c-af1c-0929e5a0ce5b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:12.530109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:12.537260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991af316-949a-4093-80ae-21f35f4212fb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:12.579827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06da404-73c8-4193-83e5-2960dce504fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:12.580470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06da404-73c8-4193-83e5-2960dce504fb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:12.580752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06da404-73c8-4193-83e5-2960dce504fb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:12.581073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06da404-73c8-4193-83e5-2960dce504fb None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:12.582214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06da404-73c8-4193-83e5-2960dce504fb None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:12.583666 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06da404-73c8-4193-83e5-2960dce504fb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:12.589991 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 63/126] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:12 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:12.589991 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06da404-73c8-4193-83e5-2960dce504fb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:12.637866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366429c0-31e0-481c-af1c-0929e5a0ce5b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:12.672260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27739612-9308-489f-a9eb-87786909a439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:12.672871 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27739612-9308-489f-a9eb-87786909a439 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:12.673438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27739612-9308-489f-a9eb-87786909a439 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:12.673565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27739612-9308-489f-a9eb-87786909a439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:12.674218 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27739612-9308-489f-a9eb-87786909a439 None admin] RBAC: Adding query filter params (role.id=671315d9acb2489cae6944e5c4c5dc77, scope.project.id=393ec3a506734c8b8c02823cb21848e2, user.id=496733ccc8ef4cfea9edc82207092046) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:12.674527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27739612-9308-489f-a9eb-87786909a439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:12.675990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27739612-9308-489f-a9eb-87786909a439 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:12.682038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 64/127] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:12 2022] GET /identity/v3/role_assignments?user.id=496733ccc8ef4cfea9edc82207092046&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=671315d9acb2489cae6944e5c4c5dc77 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:12.682941 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27739612-9308-489f-a9eb-87786909a439 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:12.700539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:12.707909 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06da404-73c8-4193-83e5-2960dce504fb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:12.750237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c1634-f90b-4a27-acb1-51321c4e2cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:12.751107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c1634-f90b-4a27-acb1-51321c4e2cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:12.751555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c1634-f90b-4a27-acb1-51321c4e2cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:12.751968 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c1634-f90b-4a27-acb1-51321c4e2cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:12.752910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c1634-f90b-4a27-acb1-51321c4e2cf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:12.753376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c1634-f90b-4a27-acb1-51321c4e2cf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:12.755042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c1634-f90b-4a27-acb1-51321c4e2cf5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:12.770831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 64/128] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:12.772010 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c1634-f90b-4a27-acb1-51321c4e2cf5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:12.801430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27739612-9308-489f-a9eb-87786909a439 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:12.808835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:12.842830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c68eb6-59ff-4733-8a29-72060907ebc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:12.843917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c68eb6-59ff-4733-8a29-72060907ebc2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:12.844031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c68eb6-59ff-4733-8a29-72060907ebc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:12.844418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c68eb6-59ff-4733-8a29-72060907ebc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:12.845222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c68eb6-59ff-4733-8a29-72060907ebc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:12.845624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c68eb6-59ff-4733-8a29-72060907ebc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:12.847296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c68eb6-59ff-4733-8a29-72060907ebc2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:12.857191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 65/129] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:12.858231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c68eb6-59ff-4733-8a29-72060907ebc2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:12.897449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c1634-f90b-4a27-acb1-51321c4e2cf5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:12.938924 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:12.945547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdd571-f27d-40ce-b688-9c928d7e36f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:12.946231 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdd571-f27d-40ce-b688-9c928d7e36f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:12.946515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdd571-f27d-40ce-b688-9c928d7e36f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:12.946813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdd571-f27d-40ce-b688-9c928d7e36f9 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2/users/7936a8595af344cb984e181520011d53/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:12.950881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdd571-f27d-40ce-b688-9c928d7e36f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=393ec3a506734c8b8c02823cb21848e2, user_id=7936a8595af344cb984e181520011d53, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:12.952334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdd571-f27d-40ce-b688-9c928d7e36f9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:12.970177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c68eb6-59ff-4733-8a29-72060907ebc2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:13.008526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b64a1-3534-453a-8bb4-e32620b40785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:13.009600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b64a1-3534-453a-8bb4-e32620b40785 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:13.010119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b64a1-3534-453a-8bb4-e32620b40785 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:13.010604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b64a1-3534-453a-8bb4-e32620b40785 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2/users/0b773d1df61d4a3b933926aeff31873c/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:13.012866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b64a1-3534-453a-8bb4-e32620b40785 None admin] RBAC: Authorizing `identity:create_grant(project_id=393ec3a506734c8b8c02823cb21848e2, user_id=0b773d1df61d4a3b933926aeff31873c, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:13.014521 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b64a1-3534-453a-8bb4-e32620b40785 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:13.031806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 65/130] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:40:12 2022] PUT /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2/users/7936a8595af344cb984e181520011d53/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:13.033092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdd571-f27d-40ce-b688-9c928d7e36f9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:13.119954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 66/131] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:40:12 2022] PUT /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2/users/0b773d1df61d4a3b933926aeff31873c/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:13.124612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b64a1-3534-453a-8bb4-e32620b40785 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:13.190248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:13.236474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdd571-f27d-40ce-b688-9c928d7e36f9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:13.298664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec109b-ba4d-45cf-8f09-d1c586d5eb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:13.299469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec109b-ba4d-45cf-8f09-d1c586d5eb27 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:13.299761 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec109b-ba4d-45cf-8f09-d1c586d5eb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:13.300044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec109b-ba4d-45cf-8f09-d1c586d5eb27 None admin] PATH_INFO: `/v3/users/0b773d1df61d4a3b933926aeff31873c` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:13.301431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec109b-ba4d-45cf-8f09-d1c586d5eb27 None admin] RBAC: Authorizing `identity:get_user(user_id=0b773d1df61d4a3b933926aeff31873c)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:13.303234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec109b-ba4d-45cf-8f09-d1c586d5eb27 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:13.305586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 66/132] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:13 2022] GET /identity/v3/users/0b773d1df61d4a3b933926aeff31873c => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:13.312590 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec109b-ba4d-45cf-8f09-d1c586d5eb27 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:13.360205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b64a1-3534-453a-8bb4-e32620b40785 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:13.404322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc7bc6-3a4d-4287-9355-c296e2bbaa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:13.404887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc7bc6-3a4d-4287-9355-c296e2bbaa8a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:13.405234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc7bc6-3a4d-4287-9355-c296e2bbaa8a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:13.405567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc7bc6-3a4d-4287-9355-c296e2bbaa8a None admin] PATH_INFO: `/v3/users/0b773d1df61d4a3b933926aeff31873c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:13.406934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc7bc6-3a4d-4287-9355-c296e2bbaa8a None admin] RBAC: Authorizing `identity:get_user(user_id=0b773d1df61d4a3b933926aeff31873c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:13.408685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc7bc6-3a4d-4287-9355-c296e2bbaa8a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:13.410523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 67/133] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:13 2022] GET /identity/v3/users/0b773d1df61d4a3b933926aeff31873c => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:13.411579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a9f38-e66c-47af-ae06-7c055ef09e83 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:13.411866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a9f38-e66c-47af-ae06-7c055ef09e83 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:13.412146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a9f38-e66c-47af-ae06-7c055ef09e83 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:13.418040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 68/134] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:13.423266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a9f38-e66c-47af-ae06-7c055ef09e83 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:13.424106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:13.450006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec109b-ba4d-45cf-8f09-d1c586d5eb27 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:13.491223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8329cd0-2124-4705-99a5-2d3daf179241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:13.492560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8329cd0-2124-4705-99a5-2d3daf179241 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:13.493210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8329cd0-2124-4705-99a5-2d3daf179241 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:13.494013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8329cd0-2124-4705-99a5-2d3daf179241 None admin] PATH_INFO: `/v3/projects/afb396faf072409096904eea3b792505` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:13.495652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8329cd0-2124-4705-99a5-2d3daf179241 None admin] RBAC: Authorizing `identity:get_project(project_id=afb396faf072409096904eea3b792505)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:13.498036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8329cd0-2124-4705-99a5-2d3daf179241 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:13.499874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 67/135] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:13 2022] GET /identity/v3/projects/afb396faf072409096904eea3b792505 => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:13.501545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ee199-d276-4703-9a4b-5036a156ea86 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:13.502035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ee199-d276-4703-9a4b-5036a156ea86 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:13.502991 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ee199-d276-4703-9a4b-5036a156ea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:13.524310 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f13ee199-d276-4703-9a4b-5036a156ea86 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:13.526108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f13ee199-d276-4703-9a4b-5036a156ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:13.543495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:13.559707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a9f38-e66c-47af-ae06-7c055ef09e83 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:13.596447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 68/136] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:13.597228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ee199-d276-4703-9a4b-5036a156ea86 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:13.622322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dfadf-2689-4da3-9e55-378b061b9f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:13.622979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dfadf-2689-4da3-9e55-378b061b9f60 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:13.623260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dfadf-2689-4da3-9e55-378b061b9f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:13.623534 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dfadf-2689-4da3-9e55-378b061b9f60 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:13.626544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dfadf-2689-4da3-9e55-378b061b9f60 None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:13.628657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dfadf-2689-4da3-9e55-378b061b9f60 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:13.631291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 69/137] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:13 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:13.633578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dfadf-2689-4da3-9e55-378b061b9f60 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:13.777831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ee199-d276-4703-9a4b-5036a156ea86 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:13.803237 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:13.817817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24763fb2-236d-4105-a412-550a2e00f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:13.818491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24763fb2-236d-4105-a412-550a2e00f214 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:13.822383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24763fb2-236d-4105-a412-550a2e00f214 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:13.822383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24763fb2-236d-4105-a412-550a2e00f214 None admin] PATH_INFO: `/v3/projects/afb396faf072409096904eea3b792505/users/0b773d1df61d4a3b933926aeff31873c/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:13.822579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24763fb2-236d-4105-a412-550a2e00f214 None admin] RBAC: Authorizing `identity:create_grant(project_id=afb396faf072409096904eea3b792505, user_id=0b773d1df61d4a3b933926aeff31873c, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:13.824382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24763fb2-236d-4105-a412-550a2e00f214 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:13.856255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dfadf-2689-4da3-9e55-378b061b9f60 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:13.894519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e598fa-bd93-454f-a224-267c091f68bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:13.895414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e598fa-bd93-454f-a224-267c091f68bf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:13.895836 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e598fa-bd93-454f-a224-267c091f68bf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:13.896297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e598fa-bd93-454f-a224-267c091f68bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:13.897619 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e598fa-bd93-454f-a224-267c091f68bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:13.899846 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e598fa-bd93-454f-a224-267c091f68bf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:13.901751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 70/138] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:13.902730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e598fa-bd93-454f-a224-267c091f68bf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:13.925572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 69/139] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:40:13 2022] PUT /identity/v3/projects/afb396faf072409096904eea3b792505/users/0b773d1df61d4a3b933926aeff31873c/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:13.926517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24763fb2-236d-4105-a412-550a2e00f214 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:13.973389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:14.085721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e598fa-bd93-454f-a224-267c091f68bf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:14.165932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abaea4-d296-41e5-95d3-e5487a915409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:14.166868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abaea4-d296-41e5-95d3-e5487a915409 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:14.167312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abaea4-d296-41e5-95d3-e5487a915409 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:14.167742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abaea4-d296-41e5-95d3-e5487a915409 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2/users/0b773d1df61d4a3b933926aeff31873c/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:14.172013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24763fb2-236d-4105-a412-550a2e00f214 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:14.173734 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abaea4-d296-41e5-95d3-e5487a915409 None admin] RBAC: Authorizing `identity:create_grant(project_id=393ec3a506734c8b8c02823cb21848e2, user_id=0b773d1df61d4a3b933926aeff31873c, role_id=cfdf14ad1f2e4a7d9eeed35bc4ffa596)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:14.175496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abaea4-d296-41e5-95d3-e5487a915409 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:14.219757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebaac6-2b23-4627-8a4c-ee5a9eaa063e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:14.220376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebaac6-2b23-4627-8a4c-ee5a9eaa063e None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:14.220662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebaac6-2b23-4627-8a4c-ee5a9eaa063e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:14.222194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebaac6-2b23-4627-8a4c-ee5a9eaa063e None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:14.222194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebaac6-2b23-4627-8a4c-ee5a9eaa063e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:14.223918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebaac6-2b23-4627-8a4c-ee5a9eaa063e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:14.261948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:14.266857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 71/140] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:40:13 2022] PUT /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2/users/0b773d1df61d4a3b933926aeff31873c/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:14.341101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 70/141] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:40:13 2022] POST /identity/v3/users => generated 283 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:40:15.138995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445c0e6-966a-4632-9a7e-c1b22d76ef4b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:15.139499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445c0e6-966a-4632-9a7e-c1b22d76ef4b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:15.139596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445c0e6-966a-4632-9a7e-c1b22d76ef4b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:15.140907 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 72/142] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:15.146500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966336-93c2-4d22-aaf1-6acfb2f27dff None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:15.146931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966336-93c2-4d22-aaf1-6acfb2f27dff None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:15.147114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b966336-93c2-4d22-aaf1-6acfb2f27dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:15.175137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7b966336-93c2-4d22-aaf1-6acfb2f27dff None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:15.176407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7b966336-93c2-4d22-aaf1-6acfb2f27dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:15.250863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:15.259072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 71/143] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:15.302450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445c0e6-966a-4632-9a7e-c1b22d76ef4b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:15.415676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445c0e6-966a-4632-9a7e-c1b22d76ef4b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:15.454025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20db0c1-35e9-4a7d-ab37-0ae1cd02cf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:15.454675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20db0c1-35e9-4a7d-ab37-0ae1cd02cf40 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:15.454958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20db0c1-35e9-4a7d-ab37-0ae1cd02cf40 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:15.455231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20db0c1-35e9-4a7d-ab37-0ae1cd02cf40 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:15.466818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20db0c1-35e9-4a7d-ab37-0ae1cd02cf40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-c20db0c1-35e9-4a7d-ab37-0ae1cd02cf40 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:15.471822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:15.472644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:15.473508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 73/144] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:15.480102 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b966336-93c2-4d22-aaf1-6acfb2f27dff None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:15.599058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b966336-93c2-4d22-aaf1-6acfb2f27dff None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:15.626474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646f0d2-eefb-472c-bd89-4d932e7a71b8 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:15.626773 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646f0d2-eefb-472c-bd89-4d932e7a71b8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:15.627028 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646f0d2-eefb-472c-bd89-4d932e7a71b8 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:15.628531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 74/145] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:15.634516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6acdfc-cc0e-492f-af03-5684cd266315 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:15.635084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6acdfc-cc0e-492f-af03-5684cd266315 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:15.635577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6acdfc-cc0e-492f-af03-5684cd266315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:15.646517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38582645-4c6f-435f-82b4-2b53efe2543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:15.647136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38582645-4c6f-435f-82b4-2b53efe2543e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:15.647424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38582645-4c6f-435f-82b4-2b53efe2543e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:15.647712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38582645-4c6f-435f-82b4-2b53efe2543e None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:15.648451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38582645-4c6f-435f-82b4-2b53efe2543e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:15.648759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38582645-4c6f-435f-82b4-2b53efe2543e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:15.650288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38582645-4c6f-435f-82b4-2b53efe2543e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:15.658310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7b6acdfc-cc0e-492f-af03-5684cd266315 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:15.659904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7b6acdfc-cc0e-492f-af03-5684cd266315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:15.665620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 72/146] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:15.666621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38582645-4c6f-435f-82b4-2b53efe2543e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:15.728918 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:15.743815 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 75/147] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:15.780642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:15.796497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6acdfc-cc0e-492f-af03-5684cd266315 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:15.924341 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38582645-4c6f-435f-82b4-2b53efe2543e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:15.986413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6acdfc-cc0e-492f-af03-5684cd266315 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:16.027927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bbcd1-2507-4794-8eab-f7063715ec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:16.028957 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bbcd1-2507-4794-8eab-f7063715ec84 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:16.029472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bbcd1-2507-4794-8eab-f7063715ec84 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:16.029948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bbcd1-2507-4794-8eab-f7063715ec84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:16.030866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd626b-d799-48a7-8489-07754f0b0da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:16.031534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd626b-d799-48a7-8489-07754f0b0da1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:16.031816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd626b-d799-48a7-8489-07754f0b0da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:16.032092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd626b-d799-48a7-8489-07754f0b0da1 None admin] PATH_INFO: `/v3/users/496733ccc8ef4cfea9edc82207092046` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:16.037817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd626b-d799-48a7-8489-07754f0b0da1 None admin] RBAC: Authorizing `identity:get_user(user_id=496733ccc8ef4cfea9edc82207092046)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:16.039352 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd626b-d799-48a7-8489-07754f0b0da1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:16.041404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 73/148] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:15 2022] GET /identity/v3/users/496733ccc8ef4cfea9edc82207092046 => generated 289 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:16.045113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350bbcd1-2507-4794-8eab-f7063715ec84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:16.051781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd626b-d799-48a7-8489-07754f0b0da1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-350bbcd1-2507-4794-8eab-f7063715ec84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:16.053863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:16.055309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:16.056056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:16.058652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 76/149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:40:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:16.066436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bbcd1-2507-4794-8eab-f7063715ec84 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:16.224358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:16.336226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd626b-d799-48a7-8489-07754f0b0da1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:16.396300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bbcd1-2507-4794-8eab-f7063715ec84 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:16.405813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:16.412497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e250b80-9a23-4a2f-87ad-098469a902a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:16.414036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e250b80-9a23-4a2f-87ad-098469a902a2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:16.414330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e250b80-9a23-4a2f-87ad-098469a902a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:16.414606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e250b80-9a23-4a2f-87ad-098469a902a2 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:16.415850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e250b80-9a23-4a2f-87ad-098469a902a2 None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:16.418201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e250b80-9a23-4a2f-87ad-098469a902a2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:16.421949 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 74/150] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:16 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:16.429555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d2fbc-e261-442c-945d-b072105b700b None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:16.429843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d2fbc-e261-442c-945d-b072105b700b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:16.430120 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d2fbc-e261-442c-945d-b072105b700b None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:16.432171 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 75/151] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:16.434786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d2fbc-e261-442c-945d-b072105b700b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:16.505888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c640ae-58de-4a1b-9ad6-9fa4a19037c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:16.506849 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c640ae-58de-4a1b-9ad6-9fa4a19037c1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:16.507862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c640ae-58de-4a1b-9ad6-9fa4a19037c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:16.508641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c640ae-58de-4a1b-9ad6-9fa4a19037c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:16.509658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c640ae-58de-4a1b-9ad6-9fa4a19037c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:16.510166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c640ae-58de-4a1b-9ad6-9fa4a19037c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:16.511746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c640ae-58de-4a1b-9ad6-9fa4a19037c1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:16.532593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 77/152] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:16.534194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bed02-e629-4581-9480-1b5466391a7a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:16.534710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bed02-e629-4581-9480-1b5466391a7a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:16.535183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bed02-e629-4581-9480-1b5466391a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:16.574018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-682bed02-e629-4581-9480-1b5466391a7a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:16.575585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-682bed02-e629-4581-9480-1b5466391a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:16.617251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d2fbc-e261-442c-945d-b072105b700b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:16.680426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f8034-41ca-4aee-8c00-d7f90e281773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:16.681118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f8034-41ca-4aee-8c00-d7f90e281773 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:16.681403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f8034-41ca-4aee-8c00-d7f90e281773 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:16.681676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f8034-41ca-4aee-8c00-d7f90e281773 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2/users/496733ccc8ef4cfea9edc82207092046/roles/671315d9acb2489cae6944e5c4c5dc77` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:16.687605 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f8034-41ca-4aee-8c00-d7f90e281773 None admin] RBAC: Authorizing `identity:create_grant(project_id=393ec3a506734c8b8c02823cb21848e2, user_id=496733ccc8ef4cfea9edc82207092046, role_id=671315d9acb2489cae6944e5c4c5dc77)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:16.689493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f8034-41ca-4aee-8c00-d7f90e281773 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:16.722747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 78/153] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:16.724036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bed02-e629-4581-9480-1b5466391a7a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:16.764180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 76/154] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:40:16 2022] PUT /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2/users/496733ccc8ef4cfea9edc82207092046/roles/671315d9acb2489cae6944e5c4c5dc77 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:16.766025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f8034-41ca-4aee-8c00-d7f90e281773 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:16.786530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:16.877894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:16.900514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bed02-e629-4581-9480-1b5466391a7a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:16.962972 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f8034-41ca-4aee-8c00-d7f90e281773 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:16.973894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91f110-e1cd-4bfc-8ea5-ea098c27d25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:16.974568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91f110-e1cd-4bfc-8ea5-ea098c27d25b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:16.974851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91f110-e1cd-4bfc-8ea5-ea098c27d25b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:16.975128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91f110-e1cd-4bfc-8ea5-ea098c27d25b None admin] PATH_INFO: `/v3/users/7936a8595af344cb984e181520011d53` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:16.979272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d91f110-e1cd-4bfc-8ea5-ea098c27d25b None admin] RBAC: Authorizing `identity:get_user(user_id=7936a8595af344cb984e181520011d53)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:16.980789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d91f110-e1cd-4bfc-8ea5-ea098c27d25b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:16.982474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 79/155] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:16 2022] GET /identity/v3/users/7936a8595af344cb984e181520011d53 => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:16.994115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91f110-e1cd-4bfc-8ea5-ea098c27d25b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:17.057781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377130d0-071d-4108-aa2f-7c48a34637a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:17.058438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377130d0-071d-4108-aa2f-7c48a34637a4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:17.058718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377130d0-071d-4108-aa2f-7c48a34637a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:17.058988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377130d0-071d-4108-aa2f-7c48a34637a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:17.089769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377130d0-071d-4108-aa2f-7c48a34637a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-377130d0-071d-4108-aa2f-7c48a34637a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:17.095100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:17.096014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:17.096768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 77/156] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:17.106070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377130d0-071d-4108-aa2f-7c48a34637a4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:17.152121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91f110-e1cd-4bfc-8ea5-ea098c27d25b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:17.168788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:17.223092 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bb201-7cf5-41aa-8ef1-41a3e715c7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:17.223729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb201-7cf5-41aa-8ef1-41a3e715c7ef None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:17.224016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb201-7cf5-41aa-8ef1-41a3e715c7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:17.224296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bb201-7cf5-41aa-8ef1-41a3e715c7ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:17.309895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377130d0-071d-4108-aa2f-7c48a34637a4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:17.336703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:17.364503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575fe5c-e527-4bd1-94dc-6d985aff1d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:17.365146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575fe5c-e527-4bd1-94dc-6d985aff1d35 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:17.365426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575fe5c-e527-4bd1-94dc-6d985aff1d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:17.365695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575fe5c-e527-4bd1-94dc-6d985aff1d35 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:17.367266 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575fe5c-e527-4bd1-94dc-6d985aff1d35 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:17.367374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575fe5c-e527-4bd1-94dc-6d985aff1d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:17.368723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575fe5c-e527-4bd1-94dc-6d985aff1d35 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:17.377422 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bb201-7cf5-41aa-8ef1-41a3e715c7ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:17.379084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bb201-7cf5-41aa-8ef1-41a3e715c7ef None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:17.382147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 78/157] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:17.387695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575fe5c-e527-4bd1-94dc-6d985aff1d35 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:17.545739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575fe5c-e527-4bd1-94dc-6d985aff1d35 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:17.580262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 80/158] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:17.587639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bb201-7cf5-41aa-8ef1-41a3e715c7ef None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:17.608558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:17.625951 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0372ab-9b1c-44bc-b305-7c9d393f002f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:17.626984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0372ab-9b1c-44bc-b305-7c9d393f002f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:17.627465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0372ab-9b1c-44bc-b305-7c9d393f002f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:17.627930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0372ab-9b1c-44bc-b305-7c9d393f002f None admin] PATH_INFO: `/v3/users/0b773d1df61d4a3b933926aeff31873c` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:17.630462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0372ab-9b1c-44bc-b305-7c9d393f002f None admin] RBAC: Authorizing `identity:get_user(user_id=0b773d1df61d4a3b933926aeff31873c)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:17.632607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0372ab-9b1c-44bc-b305-7c9d393f002f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:17.634486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 79/159] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:17 2022] GET /identity/v3/users/0b773d1df61d4a3b933926aeff31873c => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:17.640335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0372ab-9b1c-44bc-b305-7c9d393f002f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:17.719998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:17.800220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0372ab-9b1c-44bc-b305-7c9d393f002f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:17.851459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224f1fc-bc2c-43d1-978a-093bf3fca4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:17.852537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224f1fc-bc2c-43d1-978a-093bf3fca4cf None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:17.853043 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224f1fc-bc2c-43d1-978a-093bf3fca4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:17.853531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224f1fc-bc2c-43d1-978a-093bf3fca4cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:17.915247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bb201-7cf5-41aa-8ef1-41a3e715c7ef None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:17.953403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224f1fc-bc2c-43d1-978a-093bf3fca4cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:17.955540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224f1fc-bc2c-43d1-978a-093bf3fca4cf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:18.014784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:18.042309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455f079-0233-4493-98f5-fd67d43bcb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:18.042962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455f079-0233-4493-98f5-fd67d43bcb40 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:18.043242 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455f079-0233-4493-98f5-fd67d43bcb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:18.043514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455f079-0233-4493-98f5-fd67d43bcb40 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:18.044656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455f079-0233-4493-98f5-fd67d43bcb40 None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:18.046225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455f079-0233-4493-98f5-fd67d43bcb40 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:18.047866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 81/160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:17 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:18.053914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a65f0-b73e-4b8c-8576-0e8418d7d081 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:18.054371 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a65f0-b73e-4b8c-8576-0e8418d7d081 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:18.054748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a65f0-b73e-4b8c-8576-0e8418d7d081 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:18.056012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 82/161] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:18.056783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a65f0-b73e-4b8c-8576-0e8418d7d081 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:18.076359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 80/162] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:18.078885 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ffdc9-0b35-4bb4-acc8-52f3623cd450 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:18.079398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ffdc9-0b35-4bb4-acc8-52f3623cd450 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:18.079878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ffdc9-0b35-4bb4-acc8-52f3623cd450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:18.118996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-fd6ffdc9-0b35-4bb4-acc8-52f3623cd450 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:18.120109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-fd6ffdc9-0b35-4bb4-acc8-52f3623cd450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:18.236126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:18.285817 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a65f0-b73e-4b8c-8576-0e8418d7d081 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:18.306348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 81/163] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:18.307316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ffdc9-0b35-4bb4-acc8-52f3623cd450 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:18.357748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9be1a-311d-4d16-9979-e311eeab88ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:18.358436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9be1a-311d-4d16-9979-e311eeab88ff None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:18.358723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9be1a-311d-4d16-9979-e311eeab88ff None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:18.359001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9be1a-311d-4d16-9979-e311eeab88ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:18.359747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9be1a-311d-4d16-9979-e311eeab88ff None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=393ec3a506734c8b8c02823cb21848e2, user.id=7936a8595af344cb984e181520011d53) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:18.360062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9be1a-311d-4d16-9979-e311eeab88ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:18.361592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9be1a-311d-4d16-9979-e311eeab88ff None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:18.377965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 83/164] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:18 2022] GET /identity/v3/role_assignments?user.id=7936a8595af344cb984e181520011d53&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 608 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:18.377965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9be1a-311d-4d16-9979-e311eeab88ff None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:18.504499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:18.553376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ffdc9-0b35-4bb4-acc8-52f3623cd450 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:18.606064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb622b4-e002-4c3e-ad56-a2f7052cc6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:18.607577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb622b4-e002-4c3e-ad56-a2f7052cc6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:18.608076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb622b4-e002-4c3e-ad56-a2f7052cc6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:18.608547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb622b4-e002-4c3e-ad56-a2f7052cc6d2 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:18.610823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb622b4-e002-4c3e-ad56-a2f7052cc6d2 None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:18.612645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb622b4-e002-4c3e-ad56-a2f7052cc6d2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:18.614791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 82/165] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:18 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:18.616215 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430ab14-a009-4eda-964f-adca433d74b1 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:18.617276 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430ab14-a009-4eda-964f-adca433d74b1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:18.617745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430ab14-a009-4eda-964f-adca433d74b1 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:18.619564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 83/166] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:18.624187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430ab14-a009-4eda-964f-adca433d74b1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:18.652192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9be1a-311d-4d16-9979-e311eeab88ff None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:18.716962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1fd02-a0ba-4681-91d5-96527fafe17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:18.717624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1fd02-a0ba-4681-91d5-96527fafe17a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:18.717908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1fd02-a0ba-4681-91d5-96527fafe17a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:18.719151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1fd02-a0ba-4681-91d5-96527fafe17a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:18.743335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1fd02-a0ba-4681-91d5-96527fafe17a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b0c1fd02-a0ba-4681-91d5-96527fafe17a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:18.751897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:18.752770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:18.753735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 84/167] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:18.754887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead03d6-06b2-4b91-a0f2-67d0fe5efb69 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:18.755182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead03d6-06b2-4b91-a0f2-67d0fe5efb69 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:18.759096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead03d6-06b2-4b91-a0f2-67d0fe5efb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:18.775252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:18.796827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2ead03d6-06b2-4b91-a0f2-67d0fe5efb69 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:18.801872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2ead03d6-06b2-4b91-a0f2-67d0fe5efb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:18.830470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430ab14-a009-4eda-964f-adca433d74b1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:18.886622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29ec9e-83af-48f6-aec4-273100bb2d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:18.887247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29ec9e-83af-48f6-aec4-273100bb2d09 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:18.887529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29ec9e-83af-48f6-aec4-273100bb2d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:18.887804 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29ec9e-83af-48f6-aec4-273100bb2d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:18.888550 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29ec9e-83af-48f6-aec4-273100bb2d09 None admin] RBAC: Adding query filter params (role.id=bda5c75a326e4f5d9503960830898895, scope.project.id=393ec3a506734c8b8c02823cb21848e2, user.id=0b773d1df61d4a3b933926aeff31873c) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:18.888858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29ec9e-83af-48f6-aec4-273100bb2d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:18.890587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29ec9e-83af-48f6-aec4-273100bb2d09 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:18.903377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 84/168] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:18 2022] GET /identity/v3/role_assignments?user.id=0b773d1df61d4a3b933926aeff31873c&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=bda5c75a326e4f5d9503960830898895 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:18.904288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29ec9e-83af-48f6-aec4-273100bb2d09 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:18.922952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:18.967868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 85/169] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:19.015384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead03d6-06b2-4b91-a0f2-67d0fe5efb69 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:19.071196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29ec9e-83af-48f6-aec4-273100bb2d09 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:19.136644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf32db-dc61-4338-879d-920e964e05e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:19.137313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf32db-dc61-4338-879d-920e964e05e4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:19.137602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf32db-dc61-4338-879d-920e964e05e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:19.137879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf32db-dc61-4338-879d-920e964e05e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:19.138578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf32db-dc61-4338-879d-920e964e05e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:19.138887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf32db-dc61-4338-879d-920e964e05e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:19.140489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf32db-dc61-4338-879d-920e964e05e4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:19.159604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 85/170] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:19.165810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf32db-dc61-4338-879d-920e964e05e4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:19.295562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead03d6-06b2-4b91-a0f2-67d0fe5efb69 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:19.314434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:19.367493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:19.377785 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4d819-2d10-4a4c-a185-ec512611b4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:19.378475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4d819-2d10-4a4c-a185-ec512611b4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:19.379002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4d819-2d10-4a4c-a185-ec512611b4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:19.379002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4d819-2d10-4a4c-a185-ec512611b4e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:19.387141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf32db-dc61-4338-879d-920e964e05e4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:19.407515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4d819-2d10-4a4c-a185-ec512611b4e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-eea4d819-2d10-4a4c-a185-ec512611b4e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:19.419720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:19.420586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:19.421398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:19.421967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 86/171] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:19.423066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50510dc2-f542-475d-87dc-1ca365bfa826 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:19.423316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50510dc2-f542-475d-87dc-1ca365bfa826 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:19.426302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50510dc2-f542-475d-87dc-1ca365bfa826 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:19.426302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 87/172] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:19.428055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d0981-50da-45f4-86f5-92047dedb6f2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:19.428431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d0981-50da-45f4-86f5-92047dedb6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:19.428717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d0981-50da-45f4-86f5-92047dedb6f2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:19.430814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 88/173] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:19.431541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d0981-50da-45f4-86f5-92047dedb6f2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:19.454175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80908e53-6e02-41db-a255-2e7266d7e100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:19.454864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80908e53-6e02-41db-a255-2e7266d7e100 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:19.455141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80908e53-6e02-41db-a255-2e7266d7e100 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:19.455469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80908e53-6e02-41db-a255-2e7266d7e100 None admin] PATH_INFO: `/v3/users/0b773d1df61d4a3b933926aeff31873c` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:19.456696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80908e53-6e02-41db-a255-2e7266d7e100 None admin] RBAC: Authorizing `identity:get_user(user_id=0b773d1df61d4a3b933926aeff31873c)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:19.458186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80908e53-6e02-41db-a255-2e7266d7e100 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:19.459918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 86/174] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:19 2022] GET /identity/v3/users/0b773d1df61d4a3b933926aeff31873c => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:19.461778 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959737e-d137-4c1c-a073-8a503c09857c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:19.461964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959737e-d137-4c1c-a073-8a503c09857c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:19.462238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959737e-d137-4c1c-a073-8a503c09857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:19.485211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f959737e-d137-4c1c-a073-8a503c09857c None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:19.486329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f959737e-d137-4c1c-a073-8a503c09857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:19.578985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d0981-50da-45f4-86f5-92047dedb6f2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:19.584052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 87/175] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:19.590564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccdcd5-8790-4816-a45b-9355da855375 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:19.591079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccdcd5-8790-4816-a45b-9355da855375 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:19.591548 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccdcd5-8790-4816-a45b-9355da855375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:19.613992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6eccdcd5-8790-4816-a45b-9355da855375 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:19.615775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6eccdcd5-8790-4816-a45b-9355da855375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:19.639967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc786f-5c4f-44b9-8e6c-e86d0d5ca219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:19.640725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc786f-5c4f-44b9-8e6c-e86d0d5ca219 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:19.641126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc786f-5c4f-44b9-8e6c-e86d0d5ca219 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:19.641467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc786f-5c4f-44b9-8e6c-e86d0d5ca219 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:19.642262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc786f-5c4f-44b9-8e6c-e86d0d5ca219 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:19.642625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc786f-5c4f-44b9-8e6c-e86d0d5ca219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:19.644104 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc786f-5c4f-44b9-8e6c-e86d0d5ca219 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:19.655765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 89/176] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:19.656717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc786f-5c4f-44b9-8e6c-e86d0d5ca219 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:19.710879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:19.719421 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 88/177] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:19.720424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccdcd5-8790-4816-a45b-9355da855375 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:19.736031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:19.794288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc786f-5c4f-44b9-8e6c-e86d0d5ca219 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:19.833432 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b5fe9-cc48-44fd-8398-12dfa762bac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:19.834140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b5fe9-cc48-44fd-8398-12dfa762bac5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:19.834505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b5fe9-cc48-44fd-8398-12dfa762bac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:19.834846 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b5fe9-cc48-44fd-8398-12dfa762bac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:19.846131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccdcd5-8790-4816-a45b-9355da855375 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:19.882732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99404f7d-4adb-4159-8fe4-99dcc4a9eede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:19.883367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99404f7d-4adb-4159-8fe4-99dcc4a9eede None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:19.883699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99404f7d-4adb-4159-8fe4-99dcc4a9eede None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:19.883967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99404f7d-4adb-4159-8fe4-99dcc4a9eede None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:19.895389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99404f7d-4adb-4159-8fe4-99dcc4a9eede None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-99404f7d-4adb-4159-8fe4-99dcc4a9eede None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:19.901413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:19.902300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:19.903099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 89/178] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:40:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:19.904558 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99404f7d-4adb-4159-8fe4-99dcc4a9eede None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:19.915662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b5fe9-cc48-44fd-8398-12dfa762bac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:19.917167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b5fe9-cc48-44fd-8398-12dfa762bac5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:19.973493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:19.975425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:20.017166 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99404f7d-4adb-4159-8fe4-99dcc4a9eede None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:20.019992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 90/179] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:20.020973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b5fe9-cc48-44fd-8398-12dfa762bac5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:20.061350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06021ba7-bb3e-4127-a77f-02671a12ac99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:20.062016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06021ba7-bb3e-4127-a77f-02671a12ac99 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.062252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06021ba7-bb3e-4127-a77f-02671a12ac99 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.062522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06021ba7-bb3e-4127-a77f-02671a12ac99 None admin] PATH_INFO: `/v3/users/0b773d1df61d4a3b933926aeff31873c` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.063679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06021ba7-bb3e-4127-a77f-02671a12ac99 None admin] RBAC: Authorizing `identity:get_user(user_id=0b773d1df61d4a3b933926aeff31873c)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:20.065150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06021ba7-bb3e-4127-a77f-02671a12ac99 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:20.071484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 90/180] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:19 2022] GET /identity/v3/users/0b773d1df61d4a3b933926aeff31873c => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:20.071484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dbd19-7276-428b-bda0-2b4138913eb4 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.071484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dbd19-7276-428b-bda0-2b4138913eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.071484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dbd19-7276-428b-bda0-2b4138913eb4 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.071484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 91/181] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:20.071484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dbd19-7276-428b-bda0-2b4138913eb4 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:20.131581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b5fe9-cc48-44fd-8398-12dfa762bac5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:20.167681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7625bc-4e0d-4056-acab-d4630fe651dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:20.168310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7625bc-4e0d-4056-acab-d4630fe651dd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.168585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7625bc-4e0d-4056-acab-d4630fe651dd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.168853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7625bc-4e0d-4056-acab-d4630fe651dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.179103 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7625bc-4e0d-4056-acab-d4630fe651dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:20.179672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dbd19-7276-428b-bda0-2b4138913eb4 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-7c7625bc-4e0d-4056-acab-d4630fe651dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:20.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:20.184688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 91/182] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:20.189783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7625bc-4e0d-4056-acab-d4630fe651dd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:20.209034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:20.216503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:20.225185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb2792-0749-4d64-a56b-91b36a272a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:20.225773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb2792-0749-4d64-a56b-91b36a272a55 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.226042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb2792-0749-4d64-a56b-91b36a272a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.226315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb2792-0749-4d64-a56b-91b36a272a55 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.226982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb2792-0749-4d64-a56b-91b36a272a55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:20.227284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb2792-0749-4d64-a56b-91b36a272a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:20.228638 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb2792-0749-4d64-a56b-91b36a272a55 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:20.237567 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 92/183] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:20.242716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb2792-0749-4d64-a56b-91b36a272a55 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:20.302618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7625bc-4e0d-4056-acab-d4630fe651dd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:20.336626 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7286a21-0474-4490-896a-9186ae2901cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:20.337302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7286a21-0474-4490-896a-9186ae2901cf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.337562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7286a21-0474-4490-896a-9186ae2901cf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.337831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7286a21-0474-4490-896a-9186ae2901cf None admin] PATH_INFO: `/v3/projects/afb396faf072409096904eea3b792505` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.338948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7286a21-0474-4490-896a-9186ae2901cf None admin] RBAC: Authorizing `identity:get_project(project_id=afb396faf072409096904eea3b792505)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:20.340370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7286a21-0474-4490-896a-9186ae2901cf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:20.341651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb2792-0749-4d64-a56b-91b36a272a55 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:20.342020 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 92/184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:20 2022] GET /identity/v3/projects/afb396faf072409096904eea3b792505 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:20.343112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c91a-046f-46b8-8e52-76e95ee38b91 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.343389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c91a-046f-46b8-8e52-76e95ee38b91 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.343661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c91a-046f-46b8-8e52-76e95ee38b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.361149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8938c91a-046f-46b8-8e52-76e95ee38b91 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:20.362381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8938c91a-046f-46b8-8e52-76e95ee38b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:20.377190 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb65c1-9d20-438d-ad0e-06c923412e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:20.377798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb65c1-9d20-438d-ad0e-06c923412e21 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.378074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb65c1-9d20-438d-ad0e-06c923412e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.378347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb65c1-9d20-438d-ad0e-06c923412e21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.430737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 93/185] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:20.431675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938c91a-046f-46b8-8e52-76e95ee38b91 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:20.434096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:20.445957 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:20.448853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb65c1-9d20-438d-ad0e-06c923412e21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:20.450421 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb65c1-9d20-438d-ad0e-06c923412e21 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:20.531005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938c91a-046f-46b8-8e52-76e95ee38b91 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:20.549757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 93/186] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:20.554092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb65c1-9d20-438d-ad0e-06c923412e21 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:20.566533 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de5a8-fb6b-4978-9d9d-1d3c736c1132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:20.567426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de5a8-fb6b-4978-9d9d-1d3c736c1132 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.567710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de5a8-fb6b-4978-9d9d-1d3c736c1132 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.567983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de5a8-fb6b-4978-9d9d-1d3c736c1132 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.568651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1de5a8-fb6b-4978-9d9d-1d3c736c1132 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:20.568957 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1de5a8-fb6b-4978-9d9d-1d3c736c1132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:20.570400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1de5a8-fb6b-4978-9d9d-1d3c736c1132 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:20.578475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 94/187] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:20.579567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f80aa-9c7a-47bd-98da-45693496775a None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.579913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f80aa-9c7a-47bd-98da-45693496775a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.580252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f80aa-9c7a-47bd-98da-45693496775a None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.581531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 95/188] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:20.585803 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f80aa-9c7a-47bd-98da-45693496775a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:20.644518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:20.656814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb65c1-9d20-438d-ad0e-06c923412e21 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:20.659224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:20.684674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f80aa-9c7a-47bd-98da-45693496775a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:20.690710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe5e3c-1535-49ef-af55-92331e6fbda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:20.691334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe5e3c-1535-49ef-af55-92331e6fbda5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.691619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe5e3c-1535-49ef-af55-92331e6fbda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.691882 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe5e3c-1535-49ef-af55-92331e6fbda5 None admin] PATH_INFO: `/v3/users/63cfed7216bd49178aff894722414501` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.704500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe5e3c-1535-49ef-af55-92331e6fbda5 None admin] RBAC: Authorizing `identity:get_user(user_id=63cfed7216bd49178aff894722414501)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:20.705976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe5e3c-1535-49ef-af55-92331e6fbda5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:20.707570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 94/189] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:20 2022] GET /identity/v3/users/63cfed7216bd49178aff894722414501 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:20.712825 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe5e3c-1535-49ef-af55-92331e6fbda5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:20.724189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dcbe0-5dbe-4288-b2f0-b35dae76f1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:20.724804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dcbe0-5dbe-4288-b2f0-b35dae76f1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.725115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dcbe0-5dbe-4288-b2f0-b35dae76f1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.725393 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dcbe0-5dbe-4288-b2f0-b35dae76f1c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.726177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dcbe0-5dbe-4288-b2f0-b35dae76f1c5 None admin] RBAC: Adding query filter params (role.id=bda5c75a326e4f5d9503960830898895, scope.project.id=afb396faf072409096904eea3b792505, user.id=0b773d1df61d4a3b933926aeff31873c) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:20.726489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dcbe0-5dbe-4288-b2f0-b35dae76f1c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:20.727911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dcbe0-5dbe-4288-b2f0-b35dae76f1c5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:20.733741 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 96/190] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:20 2022] GET /identity/v3/role_assignments?user.id=0b773d1df61d4a3b933926aeff31873c&scope.project.id=afb396faf072409096904eea3b792505&role.id=bda5c75a326e4f5d9503960830898895 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:20.734518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dcbe0-5dbe-4288-b2f0-b35dae76f1c5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:20.836417 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe5e3c-1535-49ef-af55-92331e6fbda5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:20.845031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dcbe0-5dbe-4288-b2f0-b35dae76f1c5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:20.878930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70a9ef-1317-4401-a53d-cfb904a7f2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:20.879585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70a9ef-1317-4401-a53d-cfb904a7f2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.879847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70a9ef-1317-4401-a53d-cfb904a7f2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.880119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70a9ef-1317-4401-a53d-cfb904a7f2b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.881264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70a9ef-1317-4401-a53d-cfb904a7f2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:20.883544 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70a9ef-1317-4401-a53d-cfb904a7f2b1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:20.885271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 95/191] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:20.885726 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a738b8-ad4b-410f-b5c7-ffcbaa914ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:20.886032 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70a9ef-1317-4401-a53d-cfb904a7f2b1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:20.886892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a738b8-ad4b-410f-b5c7-ffcbaa914ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.887179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a738b8-ad4b-410f-b5c7-ffcbaa914ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.887456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a738b8-ad4b-410f-b5c7-ffcbaa914ff0 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.890096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a738b8-ad4b-410f-b5c7-ffcbaa914ff0 None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:20.891595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a738b8-ad4b-410f-b5c7-ffcbaa914ff0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:20.893758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 97/192] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:20 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:20.894601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc181714-d585-4b97-912d-f60d8041a38d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:20.894892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc181714-d585-4b97-912d-f60d8041a38d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:20.895170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc181714-d585-4b97-912d-f60d8041a38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:20.911278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:20.911455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:20.915582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fc181714-d585-4b97-912d-f60d8041a38d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:20.916779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fc181714-d585-4b97-912d-f60d8041a38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:20.995061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 98/193] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:20.995942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc181714-d585-4b97-912d-f60d8041a38d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:21.007736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70a9ef-1317-4401-a53d-cfb904a7f2b1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:21.042644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb541378-5eaf-4141-955a-5e83e1ec35ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:21.043277 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb541378-5eaf-4141-955a-5e83e1ec35ea None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:21.043558 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb541378-5eaf-4141-955a-5e83e1ec35ea None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:21.043829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb541378-5eaf-4141-955a-5e83e1ec35ea None admin] PATH_INFO: `/v3/users/63cfed7216bd49178aff894722414501` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:21.044950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb541378-5eaf-4141-955a-5e83e1ec35ea None admin] RBAC: Authorizing `identity:get_user(user_id=63cfed7216bd49178aff894722414501)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:21.046409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb541378-5eaf-4141-955a-5e83e1ec35ea None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:21.047968 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 96/194] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:20 2022] GET /identity/v3/users/63cfed7216bd49178aff894722414501 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:21.048705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb541378-5eaf-4141-955a-5e83e1ec35ea None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:21.104355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc181714-d585-4b97-912d-f60d8041a38d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:21.128508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:21.137556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c176d79-25aa-4c38-be3a-be58dfbffd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:21.140326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c176d79-25aa-4c38-be3a-be58dfbffd9d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:21.140326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c176d79-25aa-4c38-be3a-be58dfbffd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:21.140326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c176d79-25aa-4c38-be3a-be58dfbffd9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:21.142094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:21.145637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb541378-5eaf-4141-955a-5e83e1ec35ea None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:21.178212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b244b2c-5fa5-4b3c-856f-c75bee323bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:21.179505 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b244b2c-5fa5-4b3c-856f-c75bee323bdb None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:21.179875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b244b2c-5fa5-4b3c-856f-c75bee323bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:21.180149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b244b2c-5fa5-4b3c-856f-c75bee323bdb None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:21.180794 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b244b2c-5fa5-4b3c-856f-c75bee323bdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:21.182257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b244b2c-5fa5-4b3c-856f-c75bee323bdb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:21.205869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c176d79-25aa-4c38-be3a-be58dfbffd9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:21.207333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c176d79-25aa-4c38-be3a-be58dfbffd9d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:21.294255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 97/195] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:40:21 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:40:21.295082 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b244b2c-5fa5-4b3c-856f-c75bee323bdb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:21.303906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 99/196] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:21.306539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c176d79-25aa-4c38-be3a-be58dfbffd9d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:21.360811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:21.417539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b244b2c-5fa5-4b3c-856f-c75bee323bdb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:21.434424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c176d79-25aa-4c38-be3a-be58dfbffd9d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:21.457242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa1b45-2b1b-42b0-8160-c1e1a3990797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:21.457883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa1b45-2b1b-42b0-8160-c1e1a3990797 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:21.458162 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa1b45-2b1b-42b0-8160-c1e1a3990797 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:21.458434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa1b45-2b1b-42b0-8160-c1e1a3990797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:21.459169 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa1b45-2b1b-42b0-8160-c1e1a3990797 None admin] RBAC: Adding query filter params (role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596, scope.project.id=393ec3a506734c8b8c02823cb21848e2, user.id=0b773d1df61d4a3b933926aeff31873c) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:21.459474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa1b45-2b1b-42b0-8160-c1e1a3990797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:21.461507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa1b45-2b1b-42b0-8160-c1e1a3990797 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:21.467486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 98/197] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:21 2022] GET /identity/v3/role_assignments?user.id=0b773d1df61d4a3b933926aeff31873c&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:21.468372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa1b45-2b1b-42b0-8160-c1e1a3990797 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:21.471125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:21.476458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a07df7-085e-4a6f-8dcf-5f6bb6f3e745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:21.477202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a07df7-085e-4a6f-8dcf-5f6bb6f3e745 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:21.477497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a07df7-085e-4a6f-8dcf-5f6bb6f3e745 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:21.477759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a07df7-085e-4a6f-8dcf-5f6bb6f3e745 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:21.490170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a07df7-085e-4a6f-8dcf-5f6bb6f3e745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a7a07df7-085e-4a6f-8dcf-5f6bb6f3e745 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:21.495188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:21.496163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:21.496953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:21.498055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 100/198] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:21.498055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a07df7-085e-4a6f-8dcf-5f6bb6f3e745 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:21.616525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa1b45-2b1b-42b0-8160-c1e1a3990797 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:21.647035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a07df7-085e-4a6f-8dcf-5f6bb6f3e745 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:21.665063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e67b6-fd55-4610-921d-82717ef29e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:21.665736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e67b6-fd55-4610-921d-82717ef29e03 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:21.665949 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e67b6-fd55-4610-921d-82717ef29e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:21.666223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e67b6-fd55-4610-921d-82717ef29e03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:21.693489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190e127-9b64-4c3f-ab31-3b3b697c9a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:21.694139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190e127-9b64-4c3f-ab31-3b3b697c9a2c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:21.695031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190e127-9b64-4c3f-ab31-3b3b697c9a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:21.695612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190e127-9b64-4c3f-ab31-3b3b697c9a2c None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:21.696979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190e127-9b64-4c3f-ab31-3b3b697c9a2c None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:21.699035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190e127-9b64-4c3f-ab31-3b3b697c9a2c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:21.701329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 101/199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:21 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:21.702340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190e127-9b64-4c3f-ab31-3b3b697c9a2c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:21.742977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:21.758531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e67b6-fd55-4610-921d-82717ef29e03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:21.760153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e67b6-fd55-4610-921d-82717ef29e03 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:21.831574 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:21.888830 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 99/200] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:21.893431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190e127-9b64-4c3f-ab31-3b3b697c9a2c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:21.894642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e67b6-fd55-4610-921d-82717ef29e03 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:21.927899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8821d53-98b2-42fc-99c7-3e75678b5fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:21.928513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8821d53-98b2-42fc-99c7-3e75678b5fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:21.929553 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8821d53-98b2-42fc-99c7-3e75678b5fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:21.929923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8821d53-98b2-42fc-99c7-3e75678b5fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:21.930694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8821d53-98b2-42fc-99c7-3e75678b5fa7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:21.931092 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8821d53-98b2-42fc-99c7-3e75678b5fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:21.932554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8821d53-98b2-42fc-99c7-3e75678b5fa7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:21.942028 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 102/201] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:21.948604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8821d53-98b2-42fc-99c7-3e75678b5fa7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:21.979384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:22.005126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e67b6-fd55-4610-921d-82717ef29e03 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:22.039254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:22.042931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0f1c8-a445-4447-a9a7-35b16bda112b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:22.043804 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0f1c8-a445-4447-a9a7-35b16bda112b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:22.044245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0f1c8-a445-4447-a9a7-35b16bda112b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:22.044664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0f1c8-a445-4447-a9a7-35b16bda112b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:22.045670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0f1c8-a445-4447-a9a7-35b16bda112b None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6, user.id=63cfed7216bd49178aff894722414501) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:22.046213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0f1c8-a445-4447-a9a7-35b16bda112b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:22.047915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0f1c8-a445-4447-a9a7-35b16bda112b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:22.060286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8821d53-98b2-42fc-99c7-3e75678b5fa7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:22.060616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 100/202] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:21 2022] GET /identity/v3/role_assignments?user.id=63cfed7216bd49178aff894722414501&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:22.061774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0f1c8-a445-4447-a9a7-35b16bda112b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:22.100939 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100beb19-94cd-4438-8e6d-f76ab3038883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:22.102688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100beb19-94cd-4438-8e6d-f76ab3038883 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:22.103898 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100beb19-94cd-4438-8e6d-f76ab3038883 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:22.104443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100beb19-94cd-4438-8e6d-f76ab3038883 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:22.108145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100beb19-94cd-4438-8e6d-f76ab3038883 None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:22.112113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100beb19-94cd-4438-8e6d-f76ab3038883 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:22.117319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 103/203] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:21 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:22.123353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100beb19-94cd-4438-8e6d-f76ab3038883 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:22.285318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0f1c8-a445-4447-a9a7-35b16bda112b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:22.306721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:22.333710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100beb19-94cd-4438-8e6d-f76ab3038883 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:22.361739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07fbff-1fe9-4695-a373-6a1056e4c2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:22.362390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07fbff-1fe9-4695-a373-6a1056e4c2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:22.362675 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07fbff-1fe9-4695-a373-6a1056e4c2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:22.362954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07fbff-1fe9-4695-a373-6a1056e4c2a0 None admin] PATH_INFO: `/v3/users/496733ccc8ef4cfea9edc82207092046` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:22.364142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07fbff-1fe9-4695-a373-6a1056e4c2a0 None admin] RBAC: Authorizing `identity:get_user(user_id=496733ccc8ef4cfea9edc82207092046)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:22.365620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07fbff-1fe9-4695-a373-6a1056e4c2a0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:22.367775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 101/204] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:22 2022] GET /identity/v3/users/496733ccc8ef4cfea9edc82207092046 => generated 289 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:22.377939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07fbff-1fe9-4695-a373-6a1056e4c2a0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:22.385906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1b608-27a1-4506-8737-074f36563e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:22.385906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b608-27a1-4506-8737-074f36563e6c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:22.385906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b608-27a1-4506-8737-074f36563e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:22.385906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b608-27a1-4506-8737-074f36563e6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:22.386223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1b608-27a1-4506-8737-074f36563e6c None admin] RBAC: Adding query filter params (role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6, user.id=63cfed7216bd49178aff894722414501) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:22.386223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1b608-27a1-4506-8737-074f36563e6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:22.387310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1b608-27a1-4506-8737-074f36563e6c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:22.393091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 104/205] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:22 2022] GET /identity/v3/role_assignments?user.id=63cfed7216bd49178aff894722414501&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:22.429589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:22.536412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07fbff-1fe9-4695-a373-6a1056e4c2a0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:22.573829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c83174-0cc2-4067-8de2-531dc1104416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:22.574448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c83174-0cc2-4067-8de2-531dc1104416 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:22.574726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c83174-0cc2-4067-8de2-531dc1104416 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:22.574998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c83174-0cc2-4067-8de2-531dc1104416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:22.646106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c83174-0cc2-4067-8de2-531dc1104416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:22.647781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c83174-0cc2-4067-8de2-531dc1104416 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:22.687572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:22.765849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 102/206] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:22.771358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1b608-27a1-4506-8737-074f36563e6c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:22.926832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1b608-27a1-4506-8737-074f36563e6c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:22.978248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8be898-5367-4082-9286-2ae42d6f5524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:22.978890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8be898-5367-4082-9286-2ae42d6f5524 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:22.979171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8be898-5367-4082-9286-2ae42d6f5524 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:22.979445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8be898-5367-4082-9286-2ae42d6f5524 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:22.980591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8be898-5367-4082-9286-2ae42d6f5524 None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:22.982035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8be898-5367-4082-9286-2ae42d6f5524 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:22.983955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 105/207] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:22 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:22.989396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c83174-0cc2-4067-8de2-531dc1104416 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:23.156670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c83174-0cc2-4067-8de2-531dc1104416 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:23.203761 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d66a2a-d6aa-4144-b39f-520e93f0dc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:23.204586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d66a2a-d6aa-4144-b39f-520e93f0dc0e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:23.205049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d66a2a-d6aa-4144-b39f-520e93f0dc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:23.205479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d66a2a-d6aa-4144-b39f-520e93f0dc0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:23.206361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d66a2a-d6aa-4144-b39f-520e93f0dc0e None admin] RBAC: Adding query filter params (role.id=671315d9acb2489cae6944e5c4c5dc77, scope.project.id=393ec3a506734c8b8c02823cb21848e2, user.id=496733ccc8ef4cfea9edc82207092046) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:23.206830 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d66a2a-d6aa-4144-b39f-520e93f0dc0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:23.208410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d66a2a-d6aa-4144-b39f-520e93f0dc0e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:23.211150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:23.220171 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 103/208] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:22 2022] GET /identity/v3/role_assignments?user.id=496733ccc8ef4cfea9edc82207092046&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=671315d9acb2489cae6944e5c4c5dc77 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:23.805412 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b0067-2722-4a1a-aafb-0005468557cf None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:23.805964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b0067-2722-4a1a-aafb-0005468557cf None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:23.805964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b0067-2722-4a1a-aafb-0005468557cf None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:23.807575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 106/209] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:23.812497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4660ab1-d98a-4d4e-8c23-cef6781f9e0c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:23.812902 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4660ab1-d98a-4d4e-8c23-cef6781f9e0c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:23.813093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4660ab1-d98a-4d4e-8c23-cef6781f9e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:23.837296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d4660ab1-d98a-4d4e-8c23-cef6781f9e0c None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:23.838514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d4660ab1-d98a-4d4e-8c23-cef6781f9e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:23.902508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6655a-1b7f-46b0-83f2-4c8833b72826 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:23.903189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6655a-1b7f-46b0-83f2-4c8833b72826 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:23.903537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d6655a-1b7f-46b0-83f2-4c8833b72826 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:23.905217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 107/210] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:23.910105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3dd58-4b64-46cb-81f5-6ed530e85603 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:23.910466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3dd58-4b64-46cb-81f5-6ed530e85603 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:23.910819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3dd58-4b64-46cb-81f5-6ed530e85603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:23.918304 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 104/211] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:23.932450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c1d3dd58-4b64-46cb-81f5-6ed530e85603 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:23.933814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c1d3dd58-4b64-46cb-81f5-6ed530e85603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:23.943294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4660ab1-d98a-4d4e-8c23-cef6781f9e0c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:23.959311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:24.010661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 108/212] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:24.034007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3dd58-4b64-46cb-81f5-6ed530e85603 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:24.076975 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4660ab1-d98a-4d4e-8c23-cef6781f9e0c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:24.115049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:24.126254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9e183-1906-47c1-a3ae-d60900b137d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:24.127311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9e183-1906-47c1-a3ae-d60900b137d9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:24.127491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9e183-1906-47c1-a3ae-d60900b137d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:24.127721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9e183-1906-47c1-a3ae-d60900b137d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:24.128954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9e183-1906-47c1-a3ae-d60900b137d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:24.130691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9e183-1906-47c1-a3ae-d60900b137d9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:24.137049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 105/213] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:24.139270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9e183-1906-47c1-a3ae-d60900b137d9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:24.160629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3dd58-4b64-46cb-81f5-6ed530e85603 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:24.220503 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1dda58-03ab-472a-8b3f-3bdbe62fd4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:24.221461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dda58-03ab-472a-8b3f-3bdbe62fd4ec None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:24.221921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dda58-03ab-472a-8b3f-3bdbe62fd4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:24.222396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dda58-03ab-472a-8b3f-3bdbe62fd4ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:24.236979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1dda58-03ab-472a-8b3f-3bdbe62fd4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-bc1dda58-03ab-472a-8b3f-3bdbe62fd4ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:24.244713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:24.245596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:24.246344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:24.248998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 109/214] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:24.257227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9e183-1906-47c1-a3ae-d60900b137d9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:24.259189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1dda58-03ab-472a-8b3f-3bdbe62fd4ec None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:24.278586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:24.293279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59142d9c-4543-4b93-ab29-e72a502c4cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:24.294209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59142d9c-4543-4b93-ab29-e72a502c4cde None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:24.294678 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59142d9c-4543-4b93-ab29-e72a502c4cde None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:24.295131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59142d9c-4543-4b93-ab29-e72a502c4cde None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:24.295944 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59142d9c-4543-4b93-ab29-e72a502c4cde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:24.297593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59142d9c-4543-4b93-ab29-e72a502c4cde None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:24.369631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1dda58-03ab-472a-8b3f-3bdbe62fd4ec None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:24.406455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaec173-6cfe-4aa7-bcd6-6e30a24de91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:24.407072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaec173-6cfe-4aa7-bcd6-6e30a24de91a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:24.407367 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaec173-6cfe-4aa7-bcd6-6e30a24de91a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:24.407693 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaec173-6cfe-4aa7-bcd6-6e30a24de91a None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:24.414165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 106/215] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:40:24 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:40:24.414381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaec173-6cfe-4aa7-bcd6-6e30a24de91a None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:24.414381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaec173-6cfe-4aa7-bcd6-6e30a24de91a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:24.414610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e58e70-ec0b-4eef-8674-cb31fbba0bcd None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:24.414610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e58e70-ec0b-4eef-8674-cb31fbba0bcd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:24.414610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e58e70-ec0b-4eef-8674-cb31fbba0bcd None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:24.414800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaec173-6cfe-4aa7-bcd6-6e30a24de91a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:24.414874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 107/216] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:24.420614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29259232-b478-4c18-b61a-e733c7a5ffaf None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:24.420939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29259232-b478-4c18-b61a-e733c7a5ffaf None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:24.421189 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29259232-b478-4c18-b61a-e733c7a5ffaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:24.422723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 110/217] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:24.428353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaec173-6cfe-4aa7-bcd6-6e30a24de91a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:24.484267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-29259232-b478-4c18-b61a-e733c7a5ffaf None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:24.486380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-29259232-b478-4c18-b61a-e733c7a5ffaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:24.557720 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:24.573273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:24.586134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 108/218] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:24.598343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaec173-6cfe-4aa7-bcd6-6e30a24de91a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:24.608492 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29259232-b478-4c18-b61a-e733c7a5ffaf None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:24.648331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c50a9-a426-4900-b47d-74384cc9b9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:24.648970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c50a9-a426-4900-b47d-74384cc9b9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:24.651235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c50a9-a426-4900-b47d-74384cc9b9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:24.651235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c50a9-a426-4900-b47d-74384cc9b9a3 None admin] PATH_INFO: `/v3/users/e4ea30575cb447c49720d91441e2a2db` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:24.667016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c50a9-a426-4900-b47d-74384cc9b9a3 None admin] RBAC: Authorizing `identity:get_user(user_id=e4ea30575cb447c49720d91441e2a2db)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:24.668478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c50a9-a426-4900-b47d-74384cc9b9a3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:24.670358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 111/219] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:24 2022] GET /identity/v3/users/e4ea30575cb447c49720d91441e2a2db => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:24.675553 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c50a9-a426-4900-b47d-74384cc9b9a3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:24.732772 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29259232-b478-4c18-b61a-e733c7a5ffaf None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:24.781390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76f671-5530-4b9e-9434-d5d8ef7dc97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:24.781942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76f671-5530-4b9e-9434-d5d8ef7dc97b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:24.782216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76f671-5530-4b9e-9434-d5d8ef7dc97b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:24.782490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76f671-5530-4b9e-9434-d5d8ef7dc97b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:24.798020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76f671-5530-4b9e-9434-d5d8ef7dc97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-0d76f671-5530-4b9e-9434-d5d8ef7dc97b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:24.803170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:24.804500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:24.805301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:24.807689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 109/220] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:40:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:24.814840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c50a9-a426-4900-b47d-74384cc9b9a3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:24.818841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76f671-5530-4b9e-9434-d5d8ef7dc97b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:24.855694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59221769-4c85-458b-9b76-12548352390a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:24.856310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59221769-4c85-458b-9b76-12548352390a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:24.856641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59221769-4c85-458b-9b76-12548352390a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:24.856921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59221769-4c85-458b-9b76-12548352390a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:24.867073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:24.871637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:24.936081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76f671-5530-4b9e-9434-d5d8ef7dc97b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:24.939809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59221769-4c85-458b-9b76-12548352390a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:24.941566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59221769-4c85-458b-9b76-12548352390a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:24.974537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc1758-b1d2-4e87-b2f0-137a49eb5423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:24.975160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc1758-b1d2-4e87-b2f0-137a49eb5423 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:24.976082 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc1758-b1d2-4e87-b2f0-137a49eb5423 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:24.976403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc1758-b1d2-4e87-b2f0-137a49eb5423 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:24.977111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc1758-b1d2-4e87-b2f0-137a49eb5423 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:24.977722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc1758-b1d2-4e87-b2f0-137a49eb5423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:24.978916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc1758-b1d2-4e87-b2f0-137a49eb5423 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:24.988187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 110/221] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:24.995549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d441449-d201-460e-b7a2-0595c8a42afc None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:24.995549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d441449-d201-460e-b7a2-0595c8a42afc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:24.995549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d441449-d201-460e-b7a2-0595c8a42afc None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:24.995549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 111/222] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:24.999100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d441449-d201-460e-b7a2-0595c8a42afc None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:25.059833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 112/223] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 09:40:25.061572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55cbdc-8b29-47f4-af3b-38e9d4e36597 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:25.062060 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55cbdc-8b29-47f4-af3b-38e9d4e36597 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:25.062601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55cbdc-8b29-47f4-af3b-38e9d4e36597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:25.082139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9d55cbdc-8b29-47f4-af3b-38e9d4e36597 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:25.083754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9d55cbdc-8b29-47f4-af3b-38e9d4e36597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:25.112422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d441449-d201-460e-b7a2-0595c8a42afc None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:25.115035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:25.118886 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:25.156774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f4edc-0250-4608-8c20-55d55138263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:25.157468 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f4edc-0250-4608-8c20-55d55138263c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:25.159652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f4edc-0250-4608-8c20-55d55138263c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:25.159652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f4edc-0250-4608-8c20-55d55138263c None admin] PATH_INFO: `/v3/users/63cfed7216bd49178aff894722414501` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:25.160641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f4edc-0250-4608-8c20-55d55138263c None admin] RBAC: Authorizing `identity:get_user(user_id=63cfed7216bd49178aff894722414501)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:25.162248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f4edc-0250-4608-8c20-55d55138263c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:25.164416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 112/224] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:24 2022] GET /identity/v3/users/63cfed7216bd49178aff894722414501 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:25.168985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f4edc-0250-4608-8c20-55d55138263c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:25.173252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 113/225] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:25.174404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55cbdc-8b29-47f4-af3b-38e9d4e36597 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:25.287514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55cbdc-8b29-47f4-af3b-38e9d4e36597 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:25.309446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f4edc-0250-4608-8c20-55d55138263c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:25.325770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acc7cb-ac80-4d31-bcde-4c17fdc794e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:25.326749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acc7cb-ac80-4d31-bcde-4c17fdc794e5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:25.327256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acc7cb-ac80-4d31-bcde-4c17fdc794e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:25.327774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acc7cb-ac80-4d31-bcde-4c17fdc794e5 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:25.329439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74acc7cb-ac80-4d31-bcde-4c17fdc794e5 None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:25.331149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74acc7cb-ac80-4d31-bcde-4c17fdc794e5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:25.338478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 114/226] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:25 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:25.339761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acc7cb-ac80-4d31-bcde-4c17fdc794e5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:25.347115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e729f-35cc-49ec-9f15-3e95166a333d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:25.347758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e729f-35cc-49ec-9f15-3e95166a333d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:25.348041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e729f-35cc-49ec-9f15-3e95166a333d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:25.348318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e729f-35cc-49ec-9f15-3e95166a333d None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:25.349497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e729f-35cc-49ec-9f15-3e95166a333d None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:25.351396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e729f-35cc-49ec-9f15-3e95166a333d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:25.353394 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 113/227] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:25 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:25.354280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e729f-35cc-49ec-9f15-3e95166a333d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:25.382354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:25.401262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:25.457254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acc7cb-ac80-4d31-bcde-4c17fdc794e5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:25.464610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e729f-35cc-49ec-9f15-3e95166a333d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:25.496313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985850-b043-4802-9f2e-d95169ee7a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:25.497435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985850-b043-4802-9f2e-d95169ee7a70 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:25.497952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985850-b043-4802-9f2e-d95169ee7a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:25.498461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985850-b043-4802-9f2e-d95169ee7a70 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:25.502610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c21ee-6729-46e0-8890-665785bc511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:25.503278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c21ee-6729-46e0-8890-665785bc511a None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:25.503560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c21ee-6729-46e0-8890-665785bc511a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:25.503839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c21ee-6729-46e0-8890-665785bc511a None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/users/63cfed7216bd49178aff894722414501/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:25.505927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c21ee-6729-46e0-8890-665785bc511a None admin] RBAC: Authorizing `identity:create_grant(project_id=2fb27f0238c0484ca533d76cafdcc6d6, user_id=63cfed7216bd49178aff894722414501, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:25.508327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c21ee-6729-46e0-8890-665785bc511a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:25.512325 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985850-b043-4802-9f2e-d95169ee7a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e2985850-b043-4802-9f2e-d95169ee7a70 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:25.517332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:25.518205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:25.518970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 115/228] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:25.519618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985850-b043-4802-9f2e-d95169ee7a70 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:25.577537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 114/229] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:40:25 2022] PUT /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/users/63cfed7216bd49178aff894722414501/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:25.578795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c21ee-6729-46e0-8890-665785bc511a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:25.702725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:25.710842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985850-b043-4802-9f2e-d95169ee7a70 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:25.752810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8660f-b119-4a2b-8363-cce961157f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:25.753032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8660f-b119-4a2b-8363-cce961157f7d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:25.753436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8660f-b119-4a2b-8363-cce961157f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:25.753787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8660f-b119-4a2b-8363-cce961157f7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:25.754620 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8660f-b119-4a2b-8363-cce961157f7d None admin] RBAC: Adding query filter params (role.id=bda5c75a326e4f5d9503960830898895, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6, user.id=e4ea30575cb447c49720d91441e2a2db) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:25.754937 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8660f-b119-4a2b-8363-cce961157f7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:25.756453 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8660f-b119-4a2b-8363-cce961157f7d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:25.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 116/230] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:25 2022] GET /identity/v3/role_assignments?user.id=e4ea30575cb447c49720d91441e2a2db&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=bda5c75a326e4f5d9503960830898895 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:25.771960 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:25.776463 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c21ee-6729-46e0-8890-665785bc511a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:25.822007 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4247ed4-a66f-4506-9cf3-3f4230eb36fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:25.823475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4247ed4-a66f-4506-9cf3-3f4230eb36fb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:25.824005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4247ed4-a66f-4506-9cf3-3f4230eb36fb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:25.824512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4247ed4-a66f-4506-9cf3-3f4230eb36fb None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:25.825472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4247ed4-a66f-4506-9cf3-3f4230eb36fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:25.825985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4247ed4-a66f-4506-9cf3-3f4230eb36fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:25.827679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4247ed4-a66f-4506-9cf3-3f4230eb36fb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:25.839043 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 115/231] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:25.844665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8660f-b119-4a2b-8363-cce961157f7d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:25.973974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8660f-b119-4a2b-8363-cce961157f7d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:26.010855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1ad55-0ad4-4778-81e0-342d9fc88534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:26.011502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1ad55-0ad4-4778-81e0-342d9fc88534 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:26.011789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1ad55-0ad4-4778-81e0-342d9fc88534 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:26.012060 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1ad55-0ad4-4778-81e0-342d9fc88534 None admin] PATH_INFO: `/v3/users/63cfed7216bd49178aff894722414501` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:26.013320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1ad55-0ad4-4778-81e0-342d9fc88534 None admin] RBAC: Authorizing `identity:get_user(user_id=63cfed7216bd49178aff894722414501)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:26.014775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1ad55-0ad4-4778-81e0-342d9fc88534 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:26.016447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 117/232] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:25 2022] GET /identity/v3/users/63cfed7216bd49178aff894722414501 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:26.021966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4247ed4-a66f-4506-9cf3-3f4230eb36fb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:26.137915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4247ed4-a66f-4506-9cf3-3f4230eb36fb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:26.185739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac3cc3-ab2d-467c-9084-8e7d5e0f02f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:26.186399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac3cc3-ab2d-467c-9084-8e7d5e0f02f9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:26.186680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac3cc3-ab2d-467c-9084-8e7d5e0f02f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:26.186955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac3cc3-ab2d-467c-9084-8e7d5e0f02f9 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:26.188204 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac3cc3-ab2d-467c-9084-8e7d5e0f02f9 None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:26.189694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac3cc3-ab2d-467c-9084-8e7d5e0f02f9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:26.191845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 116/233] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:26 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:26.197948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1ad55-0ad4-4778-81e0-342d9fc88534 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:26.238166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:26.317720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1ad55-0ad4-4778-81e0-342d9fc88534 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:26.354956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0693474-cc15-42ec-9f23-1d615ff6dfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:26.355635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0693474-cc15-42ec-9f23-1d615ff6dfdc None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:26.355919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0693474-cc15-42ec-9f23-1d615ff6dfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:26.356253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0693474-cc15-42ec-9f23-1d615ff6dfdc None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/users/63cfed7216bd49178aff894722414501/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:26.358654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0693474-cc15-42ec-9f23-1d615ff6dfdc None admin] RBAC: Authorizing `identity:create_grant(project_id=2fb27f0238c0484ca533d76cafdcc6d6, user_id=63cfed7216bd49178aff894722414501, role_id=cfdf14ad1f2e4a7d9eeed35bc4ffa596)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:26.360099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0693474-cc15-42ec-9f23-1d615ff6dfdc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:26.466090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 118/234] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:40:26 2022] PUT /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/users/63cfed7216bd49178aff894722414501/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:26.859104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b28486-f1d4-4aa6-a8fa-6b5cf9dc46f9 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:26.859637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b28486-f1d4-4aa6-a8fa-6b5cf9dc46f9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:26.859714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b28486-f1d4-4aa6-a8fa-6b5cf9dc46f9 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:26.861096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 117/235] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:26.876651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d31f5-8a2f-439d-adee-f88164253350 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:26.877151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d31f5-8a2f-439d-adee-f88164253350 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:26.877223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d31f5-8a2f-439d-adee-f88164253350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:26.900341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ab7d31f5-8a2f-439d-adee-f88164253350 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:26.901582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ab7d31f5-8a2f-439d-adee-f88164253350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:26.961503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3a067-6db4-446d-a25b-5f287f66734b None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:26.961819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3a067-6db4-446d-a25b-5f287f66734b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:26.962111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3a067-6db4-446d-a25b-5f287f66734b None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:26.963507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 118/236] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:26.968853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d79e7-e62d-477a-ac52-87eff83cff8c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:26.969174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d79e7-e62d-477a-ac52-87eff83cff8c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:26.969449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d79e7-e62d-477a-ac52-87eff83cff8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:26.983827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 119/237] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:27.014092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-364d79e7-e62d-477a-ac52-87eff83cff8c None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:27.015384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-364d79e7-e62d-477a-ac52-87eff83cff8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:27.035509 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d31f5-8a2f-439d-adee-f88164253350 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:27.061392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:27.065344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:27.149367 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d31f5-8a2f-439d-adee-f88164253350 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:27.198526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb81a1-37ad-40b1-bb31-e5f270e4c435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:27.199182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb81a1-37ad-40b1-bb31-e5f270e4c435 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:27.199467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb81a1-37ad-40b1-bb31-e5f270e4c435 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:27.199746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fb81a1-37ad-40b1-bb31-e5f270e4c435 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:27.200974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb81a1-37ad-40b1-bb31-e5f270e4c435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:27.202611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fb81a1-37ad-40b1-bb31-e5f270e4c435 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:27.204578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 120/238] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:27.212719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb81a1-37ad-40b1-bb31-e5f270e4c435 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:27.212871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 119/239] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:27.244149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d79e7-e62d-477a-ac52-87eff83cff8c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:27.331019 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:27.353333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fb81a1-37ad-40b1-bb31-e5f270e4c435 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:27.420880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce79ba-5c27-4fc9-8cc9-7a9915dfad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:27.421922 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce79ba-5c27-4fc9-8cc9-7a9915dfad8c None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:27.422396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce79ba-5c27-4fc9-8cc9-7a9915dfad8c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:27.422879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce79ba-5c27-4fc9-8cc9-7a9915dfad8c None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:27.423807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce79ba-5c27-4fc9-8cc9-7a9915dfad8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:27.426438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce79ba-5c27-4fc9-8cc9-7a9915dfad8c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:27.447368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d79e7-e62d-477a-ac52-87eff83cff8c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:27.493343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9711b35-414c-4c27-94e7-48dd6488b834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:27.494026 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9711b35-414c-4c27-94e7-48dd6488b834 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:27.494313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9711b35-414c-4c27-94e7-48dd6488b834 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:27.494589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9711b35-414c-4c27-94e7-48dd6488b834 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:27.505334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9711b35-414c-4c27-94e7-48dd6488b834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a9711b35-414c-4c27-94e7-48dd6488b834 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:27.510316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:27.511148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:27.511904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 120/240] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:27.528149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 121/241] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:40:27 2022] POST /identity/v3/users => generated 293 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:40:27.529072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9711b35-414c-4c27-94e7-48dd6488b834 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:27.555245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:27.652682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9711b35-414c-4c27-94e7-48dd6488b834 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:27.706845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f27982-0623-44a8-89b6-7905b50308c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:27.707770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f27982-0623-44a8-89b6-7905b50308c7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:27.708219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f27982-0623-44a8-89b6-7905b50308c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:27.708658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f27982-0623-44a8-89b6-7905b50308c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:27.709597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f27982-0623-44a8-89b6-7905b50308c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:27.710693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f27982-0623-44a8-89b6-7905b50308c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:27.712302 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f27982-0623-44a8-89b6-7905b50308c7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:27.725329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 121/242] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:27.731411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce79ba-5c27-4fc9-8cc9-7a9915dfad8c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:27.864486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce79ba-5c27-4fc9-8cc9-7a9915dfad8c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:27.900086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfd224-508e-46b3-8bf6-5a3074a6a2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:27.900722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfd224-508e-46b3-8bf6-5a3074a6a2aa None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:27.901007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfd224-508e-46b3-8bf6-5a3074a6a2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:27.901308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfd224-508e-46b3-8bf6-5a3074a6a2aa None admin] PATH_INFO: `/v3/users/f4a2c9cb20254209b92d38ff9b3c5e0e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:27.912366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:27.921762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfd224-508e-46b3-8bf6-5a3074a6a2aa None admin] RBAC: Authorizing `identity:get_user(user_id=f4a2c9cb20254209b92d38ff9b3c5e0e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:27.923513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfd224-508e-46b3-8bf6-5a3074a6a2aa None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:27.925736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 122/243] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:27 2022] GET /identity/v3/users/f4a2c9cb20254209b92d38ff9b3c5e0e => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:27.932203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f27982-0623-44a8-89b6-7905b50308c7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:28.027045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:28.050378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f27982-0623-44a8-89b6-7905b50308c7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:28.092744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6d780-de4b-44b0-a81f-6acd177e0193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:28.093386 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6d780-de4b-44b0-a81f-6acd177e0193 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:28.093668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6d780-de4b-44b0-a81f-6acd177e0193 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:28.093932 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6d780-de4b-44b0-a81f-6acd177e0193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:28.136424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a21dc7-269d-47e4-a0da-44f2331d49db None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:28.136742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a21dc7-269d-47e4-a0da-44f2331d49db None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:28.136990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a21dc7-269d-47e4-a0da-44f2331d49db None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:28.138499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 123/244] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:28.143527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e595c-933c-4d5e-a6dd-bdef53f888cc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:28.143816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e595c-933c-4d5e-a6dd-bdef53f888cc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:28.144098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e595c-933c-4d5e-a6dd-bdef53f888cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:28.170994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6d780-de4b-44b0-a81f-6acd177e0193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:28.172512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6d780-de4b-44b0-a81f-6acd177e0193 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:28.173711 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-324e595c-933c-4d5e-a6dd-bdef53f888cc None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:28.175389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-324e595c-933c-4d5e-a6dd-bdef53f888cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:28.281112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 124/245] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:28.283948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:28.302056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e595c-933c-4d5e-a6dd-bdef53f888cc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:28.311208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 122/246] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:28.316873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6d780-de4b-44b0-a81f-6acd177e0193 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:28.429217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e595c-933c-4d5e-a6dd-bdef53f888cc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:28.463880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:28.468844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636ed7c-37c9-4457-ae5e-58e03de843cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:28.469616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636ed7c-37c9-4457-ae5e-58e03de843cb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:28.469899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636ed7c-37c9-4457-ae5e-58e03de843cb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:28.470234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636ed7c-37c9-4457-ae5e-58e03de843cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:28.485174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636ed7c-37c9-4457-ae5e-58e03de843cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f636ed7c-37c9-4457-ae5e-58e03de843cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:28.490075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:28.491017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:28.491840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 125/247] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:40:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:28.494183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c019edc-713a-4f85-a192-62406e68fa5c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:28.494472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c019edc-713a-4f85-a192-62406e68fa5c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:28.494747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c019edc-713a-4f85-a192-62406e68fa5c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:28.496176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 126/248] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:28.502252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c019edc-713a-4f85-a192-62406e68fa5c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:28.517850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6d780-de4b-44b0-a81f-6acd177e0193 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:28.558382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f50e8c-3140-4dfb-a303-c0c2306295c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:28.558992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50e8c-3140-4dfb-a303-c0c2306295c2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:28.559308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50e8c-3140-4dfb-a303-c0c2306295c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:28.559632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f50e8c-3140-4dfb-a303-c0c2306295c2 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:28.560949 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f50e8c-3140-4dfb-a303-c0c2306295c2 None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:28.562704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f50e8c-3140-4dfb-a303-c0c2306295c2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:28.564658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 123/249] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:28 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:28.565810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec5adb-f18e-48c2-a5b9-3aaa4c7ff111 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:28.566051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec5adb-f18e-48c2-a5b9-3aaa4c7ff111 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:28.566326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec5adb-f18e-48c2-a5b9-3aaa4c7ff111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:28.585903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-04ec5adb-f18e-48c2-a5b9-3aaa4c7ff111 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:28.587422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-04ec5adb-f18e-48c2-a5b9-3aaa4c7ff111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:28.649512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c019edc-713a-4f85-a192-62406e68fa5c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:28.680337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:28.721845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 124/250] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:28.723095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec5adb-f18e-48c2-a5b9-3aaa4c7ff111 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:28.727077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfbf30-6878-4e32-8da8-43e7cea9afca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:28.727701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfbf30-6878-4e32-8da8-43e7cea9afca None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:28.728035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfbf30-6878-4e32-8da8-43e7cea9afca None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:28.728258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfbf30-6878-4e32-8da8-43e7cea9afca None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:28.728943 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfbf30-6878-4e32-8da8-43e7cea9afca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:28.729277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfbf30-6878-4e32-8da8-43e7cea9afca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:28.730735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfbf30-6878-4e32-8da8-43e7cea9afca None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:28.753624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 127/251] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:28.756113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfbf30-6878-4e32-8da8-43e7cea9afca None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:28.839686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:28.895255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfbf30-6878-4e32-8da8-43e7cea9afca None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:28.902287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec5adb-f18e-48c2-a5b9-3aaa4c7ff111 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:28.932630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bb35c-e8d7-4afe-9852-258fb4b7690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:28.933209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bb35c-e8d7-4afe-9852-258fb4b7690e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:28.933496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bb35c-e8d7-4afe-9852-258fb4b7690e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:28.933772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707bb35c-e8d7-4afe-9852-258fb4b7690e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:28.946144 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707bb35c-e8d7-4afe-9852-258fb4b7690e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-707bb35c-e8d7-4afe-9852-258fb4b7690e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:28.952381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:28.953298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:28.954077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 128/252] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:28.954700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bb35c-e8d7-4afe-9852-258fb4b7690e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:28.989521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067c08c-01fc-45bd-8f02-f1f399db5c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:28.992052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067c08c-01fc-45bd-8f02-f1f399db5c04 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:28.992532 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067c08c-01fc-45bd-8f02-f1f399db5c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:28.992817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067c08c-01fc-45bd-8f02-f1f399db5c04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:28.993597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067c08c-01fc-45bd-8f02-f1f399db5c04 None admin] RBAC: Adding query filter params (role.id=671315d9acb2489cae6944e5c4c5dc77, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6, user.id=f4a2c9cb20254209b92d38ff9b3c5e0e) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:28.994953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067c08c-01fc-45bd-8f02-f1f399db5c04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:28.999732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067c08c-01fc-45bd-8f02-f1f399db5c04 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:29.010869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 125/253] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:28 2022] GET /identity/v3/role_assignments?user.id=f4a2c9cb20254209b92d38ff9b3c5e0e&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=671315d9acb2489cae6944e5c4c5dc77 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:29.011913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067c08c-01fc-45bd-8f02-f1f399db5c04 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:29.068715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:29.075403 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707bb35c-e8d7-4afe-9852-258fb4b7690e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:29.087223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:29.112526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cd71d-c7be-43f5-96d2-23aa794c7c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:29.113143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cd71d-c7be-43f5-96d2-23aa794c7c60 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:29.113433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cd71d-c7be-43f5-96d2-23aa794c7c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:29.113709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cd71d-c7be-43f5-96d2-23aa794c7c60 None admin] PATH_INFO: `/v3/users/63cfed7216bd49178aff894722414501` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:29.115043 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cd71d-c7be-43f5-96d2-23aa794c7c60 None admin] RBAC: Authorizing `identity:get_user(user_id=63cfed7216bd49178aff894722414501)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:29.117527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cd71d-c7be-43f5-96d2-23aa794c7c60 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:29.119320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 129/254] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:28 2022] GET /identity/v3/users/63cfed7216bd49178aff894722414501 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:29.120340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e691f79-410c-43f2-a297-c566f96ff8e3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:29.120629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e691f79-410c-43f2-a297-c566f96ff8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:29.120906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e691f79-410c-43f2-a297-c566f96ff8e3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:29.127231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 130/255] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:29.127951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e691f79-410c-43f2-a297-c566f96ff8e3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:29.142936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067c08c-01fc-45bd-8f02-f1f399db5c04 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:29.186947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3babf1-1879-4a27-ac9c-619c968b334c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:29.187551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3babf1-1879-4a27-ac9c-619c968b334c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:29.190826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3babf1-1879-4a27-ac9c-619c968b334c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:29.190826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3babf1-1879-4a27-ac9c-619c968b334c None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:29.190826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3babf1-1879-4a27-ac9c-619c968b334c None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:29.190826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3babf1-1879-4a27-ac9c-619c968b334c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:29.195364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3babf1-1879-4a27-ac9c-619c968b334c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:29.212969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 126/256] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:29.214593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e7b9-c40a-4c5a-8e9d-1355901e9428 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:29.215083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e7b9-c40a-4c5a-8e9d-1355901e9428 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:29.215560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e796e7b9-c40a-4c5a-8e9d-1355901e9428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:29.258897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e796e7b9-c40a-4c5a-8e9d-1355901e9428 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:29.260804 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e796e7b9-c40a-4c5a-8e9d-1355901e9428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:29.294072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e691f79-410c-43f2-a297-c566f96ff8e3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:29.332084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c767632-4348-4411-b1fa-3c0d9fbe8cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:29.333196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c767632-4348-4411-b1fa-3c0d9fbe8cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:29.333680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c767632-4348-4411-b1fa-3c0d9fbe8cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:29.334174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c767632-4348-4411-b1fa-3c0d9fbe8cf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:29.385085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:29.402121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 127/257] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:29.403111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796e7b9-c40a-4c5a-8e9d-1355901e9428 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:29.408185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c767632-4348-4411-b1fa-3c0d9fbe8cf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:29.411897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c767632-4348-4411-b1fa-3c0d9fbe8cf6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:29.470179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:29.520269 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 131/258] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:29.521481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c767632-4348-4411-b1fa-3c0d9fbe8cf6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:29.598391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e796e7b9-c40a-4c5a-8e9d-1355901e9428 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:29.625610 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:29.666386 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd199c0-aa67-4cb6-b827-0c732df9635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:29.666956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd199c0-aa67-4cb6-b827-0c732df9635e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:29.667234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd199c0-aa67-4cb6-b827-0c732df9635e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:29.667502 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd199c0-aa67-4cb6-b827-0c732df9635e None admin] PATH_INFO: `/v3/users/e4ea30575cb447c49720d91441e2a2db` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:29.670084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c767632-4348-4411-b1fa-3c0d9fbe8cf6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:29.672571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd199c0-aa67-4cb6-b827-0c732df9635e None admin] RBAC: Authorizing `identity:get_user(user_id=e4ea30575cb447c49720d91441e2a2db)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:29.674102 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd199c0-aa67-4cb6-b827-0c732df9635e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:29.677129 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 128/259] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:29 2022] GET /identity/v3/users/e4ea30575cb447c49720d91441e2a2db => generated 297 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:29.681955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd199c0-aa67-4cb6-b827-0c732df9635e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:29.760983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1b069-3282-434a-bd7c-16b0562e61c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:29.762066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b069-3282-434a-bd7c-16b0562e61c5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:29.762543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b069-3282-434a-bd7c-16b0562e61c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:29.763064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b069-3282-434a-bd7c-16b0562e61c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:29.788117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1b069-3282-434a-bd7c-16b0562e61c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-c6d1b069-3282-434a-bd7c-16b0562e61c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:29.793146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:29.793995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:29.794747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:29.804160 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 132/260] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:29.804941 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1b069-3282-434a-bd7c-16b0562e61c5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:29.829973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd199c0-aa67-4cb6-b827-0c732df9635e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:29.835821 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:29.906858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa7633-5963-4854-b076-9d3ae369866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:29.907492 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa7633-5963-4854-b076-9d3ae369866a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:29.908065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa7633-5963-4854-b076-9d3ae369866a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:29.908338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa7633-5963-4854-b076-9d3ae369866a None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:29.910603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa7633-5963-4854-b076-9d3ae369866a None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:29.912042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa7633-5963-4854-b076-9d3ae369866a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:29.914247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 129/261] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:29 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:29.917202 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa7633-5963-4854-b076-9d3ae369866a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:29.971271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1b069-3282-434a-bd7c-16b0562e61c5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:30.053135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa7633-5963-4854-b076-9d3ae369866a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:30.068299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0885c-35b3-4047-b637-3905b3cdf172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:30.069497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0885c-35b3-4047-b637-3905b3cdf172 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:30.069825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0885c-35b3-4047-b637-3905b3cdf172 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:30.070342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0885c-35b3-4047-b637-3905b3cdf172 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:30.077796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0885c-35b3-4047-b637-3905b3cdf172 None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:30.079268 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0885c-35b3-4047-b637-3905b3cdf172 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:30.081346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 133/262] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:29 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:30.082251 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0885c-35b3-4047-b637-3905b3cdf172 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:30.091485 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb409161-f759-44fe-99be-d2404d724f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:30.092197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb409161-f759-44fe-99be-d2404d724f88 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:30.092476 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb409161-f759-44fe-99be-d2404d724f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:30.092745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb409161-f759-44fe-99be-d2404d724f88 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:30.093444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb409161-f759-44fe-99be-d2404d724f88 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:30.093817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb409161-f759-44fe-99be-d2404d724f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:30.095331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb409161-f759-44fe-99be-d2404d724f88 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:30.099281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:30.105304 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 130/263] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:30.106146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb409161-f759-44fe-99be-d2404d724f88 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:30.159951 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:30.270121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb409161-f759-44fe-99be-d2404d724f88 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:30.283623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0885c-35b3-4047-b637-3905b3cdf172 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:30.338908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fd534-7bf8-4d12-8b80-b810a08684db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:30.339582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd534-7bf8-4d12-8b80-b810a08684db None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:30.339864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd534-7bf8-4d12-8b80-b810a08684db None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:30.340142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd534-7bf8-4d12-8b80-b810a08684db None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/users/e4ea30575cb447c49720d91441e2a2db/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:30.345595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fd534-7bf8-4d12-8b80-b810a08684db None admin] RBAC: Authorizing `identity:create_grant(project_id=2fb27f0238c0484ca533d76cafdcc6d6, user_id=e4ea30575cb447c49720d91441e2a2db, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:30.347063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fd534-7bf8-4d12-8b80-b810a08684db None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:30.351972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e625c-1463-4e13-aa1f-216788df8142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:30.352595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e625c-1463-4e13-aa1f-216788df8142 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:30.352877 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e625c-1463-4e13-aa1f-216788df8142 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:30.353176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e625c-1463-4e13-aa1f-216788df8142 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:30.353916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e625c-1463-4e13-aa1f-216788df8142 None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6, user.id=63cfed7216bd49178aff894722414501) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:30.354225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e625c-1463-4e13-aa1f-216788df8142 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:30.356964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e625c-1463-4e13-aa1f-216788df8142 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:30.364084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 134/264] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:30 2022] GET /identity/v3/role_assignments?user.id=63cfed7216bd49178aff894722414501&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:30.367241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e625c-1463-4e13-aa1f-216788df8142 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:30.421621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 131/265] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:40:30 2022] PUT /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/users/e4ea30575cb447c49720d91441e2a2db/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:30.460807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:30.503681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e625c-1463-4e13-aa1f-216788df8142 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:30.554488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708c708-e311-4340-91ad-0aeb58cb50f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:30.555144 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708c708-e311-4340-91ad-0aeb58cb50f3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:30.555423 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708c708-e311-4340-91ad-0aeb58cb50f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:30.555697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708c708-e311-4340-91ad-0aeb58cb50f3 None admin] PATH_INFO: `/v3/users/63cfed7216bd49178aff894722414501` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:30.556961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708c708-e311-4340-91ad-0aeb58cb50f3 None admin] RBAC: Authorizing `identity:get_user(user_id=63cfed7216bd49178aff894722414501)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:30.558501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708c708-e311-4340-91ad-0aeb58cb50f3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:30.560201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 135/266] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:30 2022] GET /identity/v3/users/63cfed7216bd49178aff894722414501 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:30.566192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fd534-7bf8-4d12-8b80-b810a08684db None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:30.700181 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:30.708649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fd534-7bf8-4d12-8b80-b810a08684db None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:30.746921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f90787-538a-4f8f-ba73-a6a54742253d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:30.747495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f90787-538a-4f8f-ba73-a6a54742253d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:30.747775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f90787-538a-4f8f-ba73-a6a54742253d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:30.748046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f90787-538a-4f8f-ba73-a6a54742253d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:30.751544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cf945-fa4f-4648-aee1-eb7c9c490d58 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:30.751855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cf945-fa4f-4648-aee1-eb7c9c490d58 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:30.752161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cf945-fa4f-4648-aee1-eb7c9c490d58 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:30.753874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 136/267] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:30.767787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddfe3e-b227-416f-9a2b-7b34647defb2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:30.768072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddfe3e-b227-416f-9a2b-7b34647defb2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:30.768341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddfe3e-b227-416f-9a2b-7b34647defb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:30.801149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f3ddfe3e-b227-416f-9a2b-7b34647defb2 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:30.802467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f3ddfe3e-b227-416f-9a2b-7b34647defb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:30.850665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f90787-538a-4f8f-ba73-a6a54742253d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:30.852206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f90787-538a-4f8f-ba73-a6a54742253d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:30.915982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 137/268] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:30.917990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea19121-583c-4503-9e1c-d1ad22882ffc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:30.918477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea19121-583c-4503-9e1c-d1ad22882ffc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:30.918906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea19121-583c-4503-9e1c-d1ad22882ffc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:30.921452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 138/269] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:30.926729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7627277-0781-4679-a082-7e45e2bb4fa6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:30.927233 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7627277-0781-4679-a082-7e45e2bb4fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:30.927738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7627277-0781-4679-a082-7e45e2bb4fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:30.948012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a7627277-0781-4679-a082-7e45e2bb4fa6 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:30.949584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a7627277-0781-4679-a082-7e45e2bb4fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:30.980704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:30.988293 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 132/270] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:30.989336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f90787-538a-4f8f-ba73-a6a54742253d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:31.018602 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:31.029266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 139/271] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:31.030375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7627277-0781-4679-a082-7e45e2bb4fa6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:31.099408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f90787-538a-4f8f-ba73-a6a54742253d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:31.139059 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a30e2a-98c6-4eda-b01d-b75378513ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:31.140136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a30e2a-98c6-4eda-b01d-b75378513ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:31.140555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a30e2a-98c6-4eda-b01d-b75378513ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:31.140966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a30e2a-98c6-4eda-b01d-b75378513ed4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:31.142241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a30e2a-98c6-4eda-b01d-b75378513ed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:31.143939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a30e2a-98c6-4eda-b01d-b75378513ed4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:31.146092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 133/272] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:31.147773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a30e2a-98c6-4eda-b01d-b75378513ed4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:31.158952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7627277-0781-4679-a082-7e45e2bb4fa6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:31.204991 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dccb89-f33e-4d92-9ea2-22656ea77504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:31.206149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dccb89-f33e-4d92-9ea2-22656ea77504 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:31.206683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dccb89-f33e-4d92-9ea2-22656ea77504 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:31.207159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dccb89-f33e-4d92-9ea2-22656ea77504 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:31.208612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dccb89-f33e-4d92-9ea2-22656ea77504 None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:31.210518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dccb89-f33e-4d92-9ea2-22656ea77504 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:31.212787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 140/273] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:31 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:31.213962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dccb89-f33e-4d92-9ea2-22656ea77504 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:31.229147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:31.269273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a30e2a-98c6-4eda-b01d-b75378513ed4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:31.289340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:31.305125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ea32b-8a1b-4165-91bc-e449b3fd15e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:31.305774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ea32b-8a1b-4165-91bc-e449b3fd15e1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:31.306037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ea32b-8a1b-4165-91bc-e449b3fd15e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:31.306309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ea32b-8a1b-4165-91bc-e449b3fd15e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:31.316809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ea32b-8a1b-4165-91bc-e449b3fd15e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9e5ea32b-8a1b-4165-91bc-e449b3fd15e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:31.321990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:31.322873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:31.323647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 134/274] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:31.324261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ea32b-8a1b-4165-91bc-e449b3fd15e1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:31.334024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dccb89-f33e-4d92-9ea2-22656ea77504 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:31.372834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426e9b0-9a63-4f33-915f-f4c88a06f633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:31.376839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426e9b0-9a63-4f33-915f-f4c88a06f633 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:31.376839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426e9b0-9a63-4f33-915f-f4c88a06f633 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:31.376839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426e9b0-9a63-4f33-915f-f4c88a06f633 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:31.376839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426e9b0-9a63-4f33-915f-f4c88a06f633 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:31.376839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426e9b0-9a63-4f33-915f-f4c88a06f633 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:31.436538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ea32b-8a1b-4165-91bc-e449b3fd15e1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:31.461877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:31.475638 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 141/275] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:40:31 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:40:31.475827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115ecd9-ecfc-4098-9156-cb4ea3dcc2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:31.475827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115ecd9-ecfc-4098-9156-cb4ea3dcc2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:31.475827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115ecd9-ecfc-4098-9156-cb4ea3dcc2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:31.475827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115ecd9-ecfc-4098-9156-cb4ea3dcc2f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:31.476564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115ecd9-ecfc-4098-9156-cb4ea3dcc2f2 None admin] RBAC: Adding query filter params (role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6, user.id=63cfed7216bd49178aff894722414501) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:31.476875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115ecd9-ecfc-4098-9156-cb4ea3dcc2f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:31.477968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426e9b0-9a63-4f33-915f-f4c88a06f633 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:31.478370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115ecd9-ecfc-4098-9156-cb4ea3dcc2f2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:31.489096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 135/276] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:31 2022] GET /identity/v3/role_assignments?user.id=63cfed7216bd49178aff894722414501&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:31.642553 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426e9b0-9a63-4f33-915f-f4c88a06f633 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:31.668998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:31.681971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30ef09-a1a5-4604-b20b-e7f69b9bab79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:31.684895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30ef09-a1a5-4604-b20b-e7f69b9bab79 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:31.684895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30ef09-a1a5-4604-b20b-e7f69b9bab79 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:31.684895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30ef09-a1a5-4604-b20b-e7f69b9bab79 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:31.684895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30ef09-a1a5-4604-b20b-e7f69b9bab79 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:31.684895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30ef09-a1a5-4604-b20b-e7f69b9bab79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:31.686575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30ef09-a1a5-4604-b20b-e7f69b9bab79 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:31.700212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 142/277] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:31.705514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115ecd9-ecfc-4098-9156-cb4ea3dcc2f2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:31.767301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2d2fb-4c26-4839-8015-53ef5a882be4 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:31.767624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2d2fb-4c26-4839-8015-53ef5a882be4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:31.767859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2d2fb-4c26-4839-8015-53ef5a882be4 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:31.769383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 143/278] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:31.776350 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d892f2-9e17-4476-a703-b4ba560cd787 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:31.776665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d892f2-9e17-4476-a703-b4ba560cd787 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:31.776920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d892f2-9e17-4476-a703-b4ba560cd787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:31.798533 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-66d892f2-9e17-4476-a703-b4ba560cd787 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:31.800610 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-66d892f2-9e17-4476-a703-b4ba560cd787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:31.850476 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115ecd9-ecfc-4098-9156-cb4ea3dcc2f2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:31.895257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 144/279] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:31.901518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7d1d2-49a5-4257-8048-68bd6db6f31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:31.902233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7d1d2-49a5-4257-8048-68bd6db6f31c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:31.902523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7d1d2-49a5-4257-8048-68bd6db6f31c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:31.902802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7d1d2-49a5-4257-8048-68bd6db6f31c None admin] PATH_INFO: `/v3/users/f1cde60106ca432ebb921d4118184395` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:31.917215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d892f2-9e17-4476-a703-b4ba560cd787 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:31.918202 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7d1d2-49a5-4257-8048-68bd6db6f31c None admin] RBAC: Authorizing `identity:get_user(user_id=f1cde60106ca432ebb921d4118184395)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:31.919837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7d1d2-49a5-4257-8048-68bd6db6f31c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:31.922722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 136/280] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:31 2022] GET /identity/v3/users/f1cde60106ca432ebb921d4118184395 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:31.928714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7d1d2-49a5-4257-8048-68bd6db6f31c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:31.984118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:32.032680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d892f2-9e17-4476-a703-b4ba560cd787 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:32.049353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:32.062515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7d1d2-49a5-4257-8048-68bd6db6f31c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:32.080030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c66ab-0238-4279-a312-84c8ccad64dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:32.080697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c66ab-0238-4279-a312-84c8ccad64dd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:32.080941 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c66ab-0238-4279-a312-84c8ccad64dd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:32.081237 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c66ab-0238-4279-a312-84c8ccad64dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:32.092925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c66ab-0238-4279-a312-84c8ccad64dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1e3c66ab-0238-4279-a312-84c8ccad64dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:32.098639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:32.099542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:32.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 145/281] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:32.104278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e3696-d646-47d5-b6b4-883b37c8a381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:32.104891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e3696-d646-47d5-b6b4-883b37c8a381 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:32.105226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e3696-d646-47d5-b6b4-883b37c8a381 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:32.105498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e3696-d646-47d5-b6b4-883b37c8a381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:32.106236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e3696-d646-47d5-b6b4-883b37c8a381 None admin] RBAC: Adding query filter params (role.id=bda5c75a326e4f5d9503960830898895, user.id=f1cde60106ca432ebb921d4118184395, scope.system=all) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:32.106352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c66ab-0238-4279-a312-84c8ccad64dd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:32.106602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e3696-d646-47d5-b6b4-883b37c8a381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:32.108055 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e3696-d646-47d5-b6b4-883b37c8a381 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:32.114140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 137/282] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Aug 11 09:40:31 2022] GET /identity/v3/role_assignments?user.id=f1cde60106ca432ebb921d4118184395&scope.system=all&role.id=bda5c75a326e4f5d9503960830898895 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:32.218261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c66ab-0238-4279-a312-84c8ccad64dd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:32.257368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ca34f-babb-4bda-90b3-a8260fa51489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:32.258004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ca34f-babb-4bda-90b3-a8260fa51489 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:32.258283 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ca34f-babb-4bda-90b3-a8260fa51489 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:32.258557 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ca34f-babb-4bda-90b3-a8260fa51489 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:32.259238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ca34f-babb-4bda-90b3-a8260fa51489 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:32.259541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ca34f-babb-4bda-90b3-a8260fa51489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:32.260925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ca34f-babb-4bda-90b3-a8260fa51489 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:32.270230 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 146/283] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:32.276283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e3696-d646-47d5-b6b4-883b37c8a381 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:32.413644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e3696-d646-47d5-b6b4-883b37c8a381 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:32.424472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:32.460119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d359f4-17a0-4419-afe3-cb6d799c9981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:32.460682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d359f4-17a0-4419-afe3-cb6d799c9981 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:32.460958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d359f4-17a0-4419-afe3-cb6d799c9981 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:32.461261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d359f4-17a0-4419-afe3-cb6d799c9981 None admin] PATH_INFO: `/v3/users/f4a2c9cb20254209b92d38ff9b3c5e0e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:32.465703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d359f4-17a0-4419-afe3-cb6d799c9981 None admin] RBAC: Authorizing `identity:get_user(user_id=f4a2c9cb20254209b92d38ff9b3c5e0e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:32.467163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d359f4-17a0-4419-afe3-cb6d799c9981 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:32.469106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 138/284] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:32 2022] GET /identity/v3/users/f4a2c9cb20254209b92d38ff9b3c5e0e => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:32.474624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ca34f-babb-4bda-90b3-a8260fa51489 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:32.541354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:32.636523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ca34f-babb-4bda-90b3-a8260fa51489 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:32.687919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7957b0-7111-460a-aacc-81be2b52dc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:32.688617 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7957b0-7111-460a-aacc-81be2b52dc41 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:32.688902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7957b0-7111-460a-aacc-81be2b52dc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:32.689204 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7957b0-7111-460a-aacc-81be2b52dc41 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:32.693709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7957b0-7111-460a-aacc-81be2b52dc41 None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:32.695658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7957b0-7111-460a-aacc-81be2b52dc41 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:32.697353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 147/285] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:32 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:32.703190 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d359f4-17a0-4419-afe3-cb6d799c9981 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:32.862984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d359f4-17a0-4419-afe3-cb6d799c9981 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:32.895463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f240083-0c82-466c-9585-aaefccea7007 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:32.895806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f240083-0c82-466c-9585-aaefccea7007 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:32.896012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f240083-0c82-466c-9585-aaefccea7007 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:32.898417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 148/286] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:32.906438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37677b9f-a7e1-4c50-8df0-ae5d823efecc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:32.906720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37677b9f-a7e1-4c50-8df0-ae5d823efecc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:32.906999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37677b9f-a7e1-4c50-8df0-ae5d823efecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:32.932417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-37677b9f-a7e1-4c50-8df0-ae5d823efecc None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:32.933668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-37677b9f-a7e1-4c50-8df0-ae5d823efecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:32.935651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ed810-ee25-43ae-951c-47e2c6728be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:32.936320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ed810-ee25-43ae-951c-47e2c6728be2 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:32.936599 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ed810-ee25-43ae-951c-47e2c6728be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:32.936873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ed810-ee25-43ae-951c-47e2c6728be2 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/users/f4a2c9cb20254209b92d38ff9b3c5e0e/roles/671315d9acb2489cae6944e5c4c5dc77` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:32.941311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ed810-ee25-43ae-951c-47e2c6728be2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fb27f0238c0484ca533d76cafdcc6d6, user_id=f4a2c9cb20254209b92d38ff9b3c5e0e, role_id=671315d9acb2489cae6944e5c4c5dc77)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:32.943004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ed810-ee25-43ae-951c-47e2c6728be2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:33.001741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 139/287] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:40:32 2022] PUT /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/users/f4a2c9cb20254209b92d38ff9b3c5e0e/roles/671315d9acb2489cae6944e5c4c5dc77 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:33.037445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 149/288] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:33.074192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ed810-ee25-43ae-951c-47e2c6728be2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:33.120977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:33.206191 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ed810-ee25-43ae-951c-47e2c6728be2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:33.246368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e68615-b97c-4a94-b830-f8322c0eba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:33.247004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e68615-b97c-4a94-b830-f8322c0eba09 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:33.247283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e68615-b97c-4a94-b830-f8322c0eba09 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:33.247553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e68615-b97c-4a94-b830-f8322c0eba09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:33.259410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e68615-b97c-4a94-b830-f8322c0eba09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b2e68615-b97c-4a94-b830-f8322c0eba09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:33.264265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:33.265136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:33.265907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 140/289] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:33.271641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37677b9f-a7e1-4c50-8df0-ae5d823efecc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:33.315095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:33.398015 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37677b9f-a7e1-4c50-8df0-ae5d823efecc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:33.450870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec148994-ed55-4714-b8fa-577f52c908e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:33.451507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec148994-ed55-4714-b8fa-577f52c908e5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:33.451784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec148994-ed55-4714-b8fa-577f52c908e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:33.452051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec148994-ed55-4714-b8fa-577f52c908e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:33.452739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec148994-ed55-4714-b8fa-577f52c908e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:33.453063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec148994-ed55-4714-b8fa-577f52c908e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:33.454471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec148994-ed55-4714-b8fa-577f52c908e5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:33.469344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 150/290] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:33.474602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e68615-b97c-4a94-b830-f8322c0eba09 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:33.607511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:33.616440 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e68615-b97c-4a94-b830-f8322c0eba09 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:33.658056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d8715-2754-4ba7-87bb-016f6f052713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:33.658674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d8715-2754-4ba7-87bb-016f6f052713 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:33.658963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d8715-2754-4ba7-87bb-016f6f052713 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:33.659240 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d8715-2754-4ba7-87bb-016f6f052713 None admin] PATH_INFO: `/v3/users/e4ea30575cb447c49720d91441e2a2db` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:33.665711 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d8715-2754-4ba7-87bb-016f6f052713 None admin] RBAC: Authorizing `identity:get_user(user_id=e4ea30575cb447c49720d91441e2a2db)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:33.667245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d8715-2754-4ba7-87bb-016f6f052713 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:33.668850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 141/291] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:33 2022] GET /identity/v3/users/e4ea30575cb447c49720d91441e2a2db => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:33.674368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec148994-ed55-4714-b8fa-577f52c908e5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:33.824187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:33.831240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec148994-ed55-4714-b8fa-577f52c908e5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:33.883145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58f3ff-1754-435a-b066-87dd2bbe854d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:33.883762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58f3ff-1754-435a-b066-87dd2bbe854d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:33.884040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58f3ff-1754-435a-b066-87dd2bbe854d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:33.884309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58f3ff-1754-435a-b066-87dd2bbe854d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:33.978312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58f3ff-1754-435a-b066-87dd2bbe854d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:33.979690 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58f3ff-1754-435a-b066-87dd2bbe854d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:34.107507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:34.115851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 151/292] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:34.121595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d8715-2754-4ba7-87bb-016f6f052713 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:34.272855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d8715-2754-4ba7-87bb-016f6f052713 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:34.313857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cfa2e-5309-4bfb-b0eb-d0f626c282a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:34.314497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cfa2e-5309-4bfb-b0eb-d0f626c282a6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:34.314781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cfa2e-5309-4bfb-b0eb-d0f626c282a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:34.315060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cfa2e-5309-4bfb-b0eb-d0f626c282a6 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:34.321775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cfa2e-5309-4bfb-b0eb-d0f626c282a6 None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:34.323338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cfa2e-5309-4bfb-b0eb-d0f626c282a6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:34.325088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 142/293] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:34 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:34.330850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58f3ff-1754-435a-b066-87dd2bbe854d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:34.375967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad8528-f868-40fe-977a-b8578c06a077 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:34.382395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad8528-f868-40fe-977a-b8578c06a077 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:34.382395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad8528-f868-40fe-977a-b8578c06a077 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:34.382395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 143/294] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:34.387648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99f712-81b6-4103-8773-05b36dc7fa47 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:34.388034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99f712-81b6-4103-8773-05b36dc7fa47 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:34.388406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99f712-81b6-4103-8773-05b36dc7fa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:34.479176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-fb99f712-81b6-4103-8773-05b36dc7fa47 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:34.480383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-fb99f712-81b6-4103-8773-05b36dc7fa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:34.498708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58f3ff-1754-435a-b066-87dd2bbe854d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:34.541787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b885ae1-52c4-4ec1-8761-963f6d9d87b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:34.542755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b885ae1-52c4-4ec1-8761-963f6d9d87b3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:34.544387 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b885ae1-52c4-4ec1-8761-963f6d9d87b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:34.544860 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b885ae1-52c4-4ec1-8761-963f6d9d87b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:34.545365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:34.546176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b885ae1-52c4-4ec1-8761-963f6d9d87b3 None admin] RBAC: Adding query filter params (role.id=bda5c75a326e4f5d9503960830898895, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6, user.id=e4ea30575cb447c49720d91441e2a2db) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:34.546176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b885ae1-52c4-4ec1-8761-963f6d9d87b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:34.550941 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b885ae1-52c4-4ec1-8761-963f6d9d87b3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:34.566247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 152/295] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:34 2022] GET /identity/v3/role_assignments?user.id=e4ea30575cb447c49720d91441e2a2db&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=bda5c75a326e4f5d9503960830898895 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:34.613147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 144/296] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:34.639457 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341de8a-77c8-443d-9e96-b84b12ec2037 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:34.640068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341de8a-77c8-443d-9e96-b84b12ec2037 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:34.640568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341de8a-77c8-443d-9e96-b84b12ec2037 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:34.642525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 153/297] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:34.648727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a18a7-60a6-47fc-995d-fdd462e85d6d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:34.649375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a18a7-60a6-47fc-995d-fdd462e85d6d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:34.649845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a18a7-60a6-47fc-995d-fdd462e85d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:34.654709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341de8a-77c8-443d-9e96-b84b12ec2037 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:34.690478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-226a18a7-60a6-47fc-995d-fdd462e85d6d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:34.691770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-226a18a7-60a6-47fc-995d-fdd462e85d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:34.758805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:34.798048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 145/298] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:34.799674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7b184-9468-4c36-b751-f37117727359 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:34.800175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7b184-9468-4c36-b751-f37117727359 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:34.800637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c7b184-9468-4c36-b751-f37117727359 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:34.802090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 146/299] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:34.808370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb8819-50e0-4740-9e60-ee3e446eb010 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:34.809078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb8819-50e0-4740-9e60-ee3e446eb010 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:34.809886 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb8819-50e0-4740-9e60-ee3e446eb010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:34.813460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341de8a-77c8-443d-9e96-b84b12ec2037 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:34.831539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3ebb8819-50e0-4740-9e60-ee3e446eb010 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:34.833030 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3ebb8819-50e0-4740-9e60-ee3e446eb010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:34.852969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd40a36-0de4-4d49-8422-b47123d53840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:34.853630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd40a36-0de4-4d49-8422-b47123d53840 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:34.853992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd40a36-0de4-4d49-8422-b47123d53840 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:34.854165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd40a36-0de4-4d49-8422-b47123d53840 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:34.866870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd40a36-0de4-4d49-8422-b47123d53840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0fd40a36-0de4-4d49-8422-b47123d53840 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:34.872284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:34.873184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:34.873962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 154/300] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:34.874969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd40a36-0de4-4d49-8422-b47123d53840 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:34.913518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 147/301] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:34.914477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb8819-50e0-4740-9e60-ee3e446eb010 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:34.944088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:34.994761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd40a36-0de4-4d49-8422-b47123d53840 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:35.015617 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:35.030158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375104b-e9ad-4385-aa3a-062400659946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:35.030791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375104b-e9ad-4385-aa3a-062400659946 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.031074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375104b-e9ad-4385-aa3a-062400659946 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.031273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb8819-50e0-4740-9e60-ee3e446eb010 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:35.031374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375104b-e9ad-4385-aa3a-062400659946 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:35.032440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375104b-e9ad-4385-aa3a-062400659946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:35.034172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375104b-e9ad-4385-aa3a-062400659946 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:35.035559 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 155/302] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:35.041297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375104b-e9ad-4385-aa3a-062400659946 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:35.072503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31daa3a-f052-4bfe-af62-3a051051f537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:35.073356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31daa3a-f052-4bfe-af62-3a051051f537 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.073823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31daa3a-f052-4bfe-af62-3a051051f537 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.074183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31daa3a-f052-4bfe-af62-3a051051f537 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:35.074768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31daa3a-f052-4bfe-af62-3a051051f537 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:35.075073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31daa3a-f052-4bfe-af62-3a051051f537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:35.076450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31daa3a-f052-4bfe-af62-3a051051f537 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:35.085605 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 148/303] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:35.090854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31daa3a-f052-4bfe-af62-3a051051f537 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:35.148174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375104b-e9ad-4385-aa3a-062400659946 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:35.176141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:35.190019 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb1530-198b-47a6-9318-09ea5cfbd6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:35.190595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb1530-198b-47a6-9318-09ea5cfbd6bd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.190880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb1530-198b-47a6-9318-09ea5cfbd6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.191150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb1530-198b-47a6-9318-09ea5cfbd6bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:35.198585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31daa3a-f052-4bfe-af62-3a051051f537 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:35.202731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb1530-198b-47a6-9318-09ea5cfbd6bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-05bb1530-198b-47a6-9318-09ea5cfbd6bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:35.207976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:35.209001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:35.209909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 156/304] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:35.210723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb1530-198b-47a6-9318-09ea5cfbd6bd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:35.239712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca35d7a-eea0-4c34-81fa-67c3555376ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:35.240841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca35d7a-eea0-4c34-81fa-67c3555376ba None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.241268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca35d7a-eea0-4c34-81fa-67c3555376ba None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.241648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca35d7a-eea0-4c34-81fa-67c3555376ba None admin] PATH_INFO: `/v3/groups` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:35.242543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca35d7a-eea0-4c34-81fa-67c3555376ba None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:35.244329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca35d7a-eea0-4c34-81fa-67c3555376ba None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:35.264663 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:35.314362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb1530-198b-47a6-9318-09ea5cfbd6bd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:35.325941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 149/305] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Aug 11 09:40:35 2022] POST /identity/v3/groups => generated 229 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:40:35.325941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca35d7a-eea0-4c34-81fa-67c3555376ba None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:35.353055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a778d-b205-4b75-bc51-5e04ae613751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:35.354076 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a778d-b205-4b75-bc51-5e04ae613751 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.354543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a778d-b205-4b75-bc51-5e04ae613751 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.355008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a778d-b205-4b75-bc51-5e04ae613751 None admin] PATH_INFO: `/v3/users/b6fecb2bde1f4ea987c53ee2525c1ad7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:35.372714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a778d-b205-4b75-bc51-5e04ae613751 None admin] RBAC: Authorizing `identity:get_user(user_id=b6fecb2bde1f4ea987c53ee2525c1ad7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:35.374208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a778d-b205-4b75-bc51-5e04ae613751 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:35.381761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 157/306] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:35 2022] GET /identity/v3/users/b6fecb2bde1f4ea987c53ee2525c1ad7 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:35.382561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a778d-b205-4b75-bc51-5e04ae613751 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:35.475686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca35d7a-eea0-4c34-81fa-67c3555376ba None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:35.513712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a778d-b205-4b75-bc51-5e04ae613751 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:35.514830 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9859625-55af-484d-94f1-90105bd5d069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:35.515670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9859625-55af-484d-94f1-90105bd5d069 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.516221 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9859625-55af-484d-94f1-90105bd5d069 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.516692 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9859625-55af-484d-94f1-90105bd5d069 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:35.517599 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9859625-55af-484d-94f1-90105bd5d069 None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:35.518101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9859625-55af-484d-94f1-90105bd5d069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:35.519707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9859625-55af-484d-94f1-90105bd5d069 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:35.522293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:35.531403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 150/307] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:35.536846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9859625-55af-484d-94f1-90105bd5d069 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:35.547392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:35.551573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1bb38-24ac-4169-9526-2a7977c5f5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:35.552183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1bb38-24ac-4169-9526-2a7977c5f5cf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.552462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1bb38-24ac-4169-9526-2a7977c5f5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.552731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1bb38-24ac-4169-9526-2a7977c5f5cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:35.553678 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1bb38-24ac-4169-9526-2a7977c5f5cf None admin] RBAC: Adding query filter params (role.id=671315d9acb2489cae6944e5c4c5dc77, user.id=b6fecb2bde1f4ea987c53ee2525c1ad7, scope.system=all) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:35.553989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1bb38-24ac-4169-9526-2a7977c5f5cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:35.555411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1bb38-24ac-4169-9526-2a7977c5f5cf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:35.560972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 158/308] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Aug 11 09:40:35 2022] GET /identity/v3/role_assignments?user.id=b6fecb2bde1f4ea987c53ee2525c1ad7&scope.system=all&role.id=671315d9acb2489cae6944e5c4c5dc77 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:35.658517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9859625-55af-484d-94f1-90105bd5d069 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:35.671445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e6ee1-38c6-455c-9b0d-d68a455b1c2c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.671887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e6ee1-38c6-455c-9b0d-d68a455b1c2c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.672260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e6ee1-38c6-455c-9b0d-d68a455b1c2c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:35.674694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 159/309] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:35.683818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71ce58-6ba4-4b00-8098-ec863050499d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.684264 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71ce58-6ba4-4b00-8098-ec863050499d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.684668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71ce58-6ba4-4b00-8098-ec863050499d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:35.704920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb5c11-a74f-4b57-b7a2-4192d2531bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:35.705876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5c11-a74f-4b57-b7a2-4192d2531bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.706294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5c11-a74f-4b57-b7a2-4192d2531bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.706703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5c11-a74f-4b57-b7a2-4192d2531bf9 None admin] PATH_INFO: `/v3/users/f1cde60106ca432ebb921d4118184395` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:35.708056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb5c11-a74f-4b57-b7a2-4192d2531bf9 None admin] RBAC: Authorizing `identity:get_user(user_id=f1cde60106ca432ebb921d4118184395)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:35.708810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4a71ce58-6ba4-4b00-8098-ec863050499d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:35.710626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb5c11-a74f-4b57-b7a2-4192d2531bf9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:35.710746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4a71ce58-6ba4-4b00-8098-ec863050499d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:35.712531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 151/310] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:35 2022] GET /identity/v3/users/f1cde60106ca432ebb921d4118184395 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:35.718325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb5c11-a74f-4b57-b7a2-4192d2531bf9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:35.805478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:35.813120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 160/311] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:35.833589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71ce58-6ba4-4b00-8098-ec863050499d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:35.849274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb5c11-a74f-4b57-b7a2-4192d2531bf9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:35.892589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385135d-c9f4-481c-bf72-3ff622c4a106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:35.893782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385135d-c9f4-481c-bf72-3ff622c4a106 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.894065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385135d-c9f4-481c-bf72-3ff622c4a106 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.894343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385135d-c9f4-481c-bf72-3ff622c4a106 None admin] PATH_INFO: `/v3/system/users/f1cde60106ca432ebb921d4118184395/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:35.896762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385135d-c9f4-481c-bf72-3ff622c4a106 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f1cde60106ca432ebb921d4118184395, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:35.898811 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385135d-c9f4-481c-bf72-3ff622c4a106 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:35.949087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:35.952006 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71ce58-6ba4-4b00-8098-ec863050499d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:35.967667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 152/312] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Aug 11 09:40:35 2022] PUT /identity/v3/system/users/f1cde60106ca432ebb921d4118184395/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 09:40:35.996718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed27faa-84f4-4425-a8d1-e09701708215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:35.997731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed27faa-84f4-4425-a8d1-e09701708215 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:35.998030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed27faa-84f4-4425-a8d1-e09701708215 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:35.998312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed27faa-84f4-4425-a8d1-e09701708215 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:36.012149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed27faa-84f4-4425-a8d1-e09701708215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-7ed27faa-84f4-4425-a8d1-e09701708215 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:36.017355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:36.018278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:36.019290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 161/313] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:36.027086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385135d-c9f4-481c-bf72-3ff622c4a106 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:36.140602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385135d-c9f4-481c-bf72-3ff622c4a106 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:36.176436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a5274-86a5-45ad-b893-ba4004507666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:36.177110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a5274-86a5-45ad-b893-ba4004507666 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:36.177353 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a5274-86a5-45ad-b893-ba4004507666 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:36.177612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a5274-86a5-45ad-b893-ba4004507666 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:36.178478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a5274-86a5-45ad-b893-ba4004507666 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:36.178478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a5274-86a5-45ad-b893-ba4004507666 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:36.180419 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a5274-86a5-45ad-b893-ba4004507666 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:36.187797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:36.195061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 153/314] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:36.196578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed27faa-84f4-4425-a8d1-e09701708215 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:36.304672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed27faa-84f4-4425-a8d1-e09701708215 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:36.341045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ade289-919c-4c50-a81f-5865599cffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:36.341691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ade289-919c-4c50-a81f-5865599cffaa None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:36.341976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ade289-919c-4c50-a81f-5865599cffaa None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:36.342249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ade289-919c-4c50-a81f-5865599cffaa None admin] PATH_INFO: `/v3/users/f4a2c9cb20254209b92d38ff9b3c5e0e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:36.343398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ade289-919c-4c50-a81f-5865599cffaa None admin] RBAC: Authorizing `identity:get_user(user_id=f4a2c9cb20254209b92d38ff9b3c5e0e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:36.344844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ade289-919c-4c50-a81f-5865599cffaa None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:36.346796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 162/315] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:36 2022] GET /identity/v3/users/f4a2c9cb20254209b92d38ff9b3c5e0e => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:36.352186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a5274-86a5-45ad-b893-ba4004507666 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:36.494946 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a5274-86a5-45ad-b893-ba4004507666 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:36.529791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e973b-0f9e-43c2-83c5-d72ce1640e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:36.530404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e973b-0f9e-43c2-83c5-d72ce1640e0b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:36.530687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e973b-0f9e-43c2-83c5-d72ce1640e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:36.530965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e973b-0f9e-43c2-83c5-d72ce1640e0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:36.607370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:36.610041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e973b-0f9e-43c2-83c5-d72ce1640e0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:36.611512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e973b-0f9e-43c2-83c5-d72ce1640e0b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:36.731125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 154/316] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:36.736625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ade289-919c-4c50-a81f-5865599cffaa None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:36.770939 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:36.847114 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ade289-919c-4c50-a81f-5865599cffaa None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:36.882135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeec2da-fc1d-458c-a0d1-c29240b2ea9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:36.882707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeec2da-fc1d-458c-a0d1-c29240b2ea9f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:36.882991 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeec2da-fc1d-458c-a0d1-c29240b2ea9f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:36.883268 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeec2da-fc1d-458c-a0d1-c29240b2ea9f None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:36.884416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeec2da-fc1d-458c-a0d1-c29240b2ea9f None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:36.885861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeec2da-fc1d-458c-a0d1-c29240b2ea9f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:36.892331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 163/317] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:36 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:36.893727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e973b-0f9e-43c2-83c5-d72ce1640e0b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:36.998624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:37.002503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e973b-0f9e-43c2-83c5-d72ce1640e0b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:37.039268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bd1d0-53c3-4032-bd91-09f914cfb970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:37.040110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bd1d0-53c3-4032-bd91-09f914cfb970 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:37.040535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bd1d0-53c3-4032-bd91-09f914cfb970 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:37.040979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bd1d0-53c3-4032-bd91-09f914cfb970 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:37.041879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802bd1d0-53c3-4032-bd91-09f914cfb970 None admin] RBAC: Adding query filter params (role.id=671315d9acb2489cae6944e5c4c5dc77, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6, user.id=f4a2c9cb20254209b92d38ff9b3c5e0e) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:37.042328 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802bd1d0-53c3-4032-bd91-09f914cfb970 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:37.043965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802bd1d0-53c3-4032-bd91-09f914cfb970 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:37.050373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 155/318] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:40:36 2022] GET /identity/v3/role_assignments?user.id=f4a2c9cb20254209b92d38ff9b3c5e0e&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=671315d9acb2489cae6944e5c4c5dc77 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:37.524622 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa1c3c-08f2-469d-8595-e4e8d5948c05 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:37.525170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa1c3c-08f2-469d-8595-e4e8d5948c05 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:37.525250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa1c3c-08f2-469d-8595-e4e8d5948c05 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:37.526687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 164/319] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:37.531540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28630e3b-0e75-4c8f-b90b-ff91cf881f5d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:37.531939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28630e3b-0e75-4c8f-b90b-ff91cf881f5d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:37.532114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28630e3b-0e75-4c8f-b90b-ff91cf881f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:37.549930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-28630e3b-0e75-4c8f-b90b-ff91cf881f5d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:37.551120 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-28630e3b-0e75-4c8f-b90b-ff91cf881f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:37.591824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed2a7c-15ad-4b7c-a30a-b047ce4265c4 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:37.592149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed2a7c-15ad-4b7c-a30a-b047ce4265c4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:37.592378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed2a7c-15ad-4b7c-a30a-b047ce4265c4 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:37.593894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 165/320] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:37.598459 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918abd9-98bf-49bb-bca3-9f438282fc63 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:37.598743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918abd9-98bf-49bb-bca3-9f438282fc63 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:37.599018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918abd9-98bf-49bb-bca3-9f438282fc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:37.616322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d918abd9-98bf-49bb-bca3-9f438282fc63 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:37.617534 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d918abd9-98bf-49bb-bca3-9f438282fc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:37.621817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 156/321] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:37.642338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28630e3b-0e75-4c8f-b90b-ff91cf881f5d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:37.691318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 166/322] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:37.714414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918abd9-98bf-49bb-bca3-9f438282fc63 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:37.740064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:37.746810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28630e3b-0e75-4c8f-b90b-ff91cf881f5d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:37.747460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:37.783633 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f11e9-155c-4fa5-a252-f53c531754ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:37.784192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f11e9-155c-4fa5-a252-f53c531754ca None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:37.784475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f11e9-155c-4fa5-a252-f53c531754ca None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:37.784749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f11e9-155c-4fa5-a252-f53c531754ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:37.785918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f11e9-155c-4fa5-a252-f53c531754ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:37.787571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f11e9-155c-4fa5-a252-f53c531754ca None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:37.789327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 157/323] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:37.795303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f11e9-155c-4fa5-a252-f53c531754ca None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:37.824386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918abd9-98bf-49bb-bca3-9f438282fc63 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:37.864036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18452a-3d42-4f67-881a-f1b4b146d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:37.864670 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18452a-3d42-4f67-881a-f1b4b146d8fa None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:37.864961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18452a-3d42-4f67-881a-f1b4b146d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:37.865262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18452a-3d42-4f67-881a-f1b4b146d8fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:37.878687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18452a-3d42-4f67-881a-f1b4b146d8fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0e18452a-3d42-4f67-881a-f1b4b146d8fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:37.884125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:37.885213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:37.886072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 167/324] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:37.890987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18452a-3d42-4f67-881a-f1b4b146d8fa None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:37.902900 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f11e9-155c-4fa5-a252-f53c531754ca None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:37.939582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85eb0f0-710a-4b0a-ba72-c97a5a7c9fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:37.940416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85eb0f0-710a-4b0a-ba72-c97a5a7c9fc1 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:37.940850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85eb0f0-710a-4b0a-ba72-c97a5a7c9fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:37.941405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85eb0f0-710a-4b0a-ba72-c97a5a7c9fc1 None admin] PATH_INFO: `/v3/groups` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:37.942528 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85eb0f0-710a-4b0a-ba72-c97a5a7c9fc1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:37.944118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85eb0f0-710a-4b0a-ba72-c97a5a7c9fc1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:37.980577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:37.991962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18452a-3d42-4f67-881a-f1b4b146d8fa None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:38.024603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49460c1-b9ba-4808-adb9-f247a26c4bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:38.025239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49460c1-b9ba-4808-adb9-f247a26c4bee None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:38.025521 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49460c1-b9ba-4808-adb9-f247a26c4bee None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:38.025793 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49460c1-b9ba-4808-adb9-f247a26c4bee None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:38.026455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49460c1-b9ba-4808-adb9-f247a26c4bee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:38.026757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49460c1-b9ba-4808-adb9-f247a26c4bee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:38.028240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49460c1-b9ba-4808-adb9-f247a26c4bee None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:38.036114 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 168/325] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:38.042376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49460c1-b9ba-4808-adb9-f247a26c4bee None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:38.051616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 158/326] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Aug 11 09:40:37 2022] POST /identity/v3/groups => generated 226 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:40:38.166081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49460c1-b9ba-4808-adb9-f247a26c4bee None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:38.192869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f81d6-cb35-4e09-bfc4-fc33dbac7176 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:38.193175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f81d6-cb35-4e09-bfc4-fc33dbac7176 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:38.193447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f81d6-cb35-4e09-bfc4-fc33dbac7176 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:38.196437 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 159/327] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:38.201969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f535510-7190-42eb-9375-f2cdf6839bba None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:38.202242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f535510-7190-42eb-9375-f2cdf6839bba None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:38.202559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f535510-7190-42eb-9375-f2cdf6839bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:38.216023 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbabe9c-787a-465d-845a-a3b08740d090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:38.216666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbabe9c-787a-465d-845a-a3b08740d090 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:38.217027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbabe9c-787a-465d-845a-a3b08740d090 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:38.217260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbabe9c-787a-465d-845a-a3b08740d090 None admin] PATH_INFO: `/v3/users/b6fecb2bde1f4ea987c53ee2525c1ad7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:38.218956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbabe9c-787a-465d-845a-a3b08740d090 None admin] RBAC: Authorizing `identity:get_user(user_id=b6fecb2bde1f4ea987c53ee2525c1ad7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:38.220389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbabe9c-787a-465d-845a-a3b08740d090 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:38.222049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 169/328] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:38 2022] GET /identity/v3/users/b6fecb2bde1f4ea987c53ee2525c1ad7 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:38.226568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1f535510-7190-42eb-9375-f2cdf6839bba None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:38.227436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1f535510-7190-42eb-9375-f2cdf6839bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:38.228084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbabe9c-787a-465d-845a-a3b08740d090 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:38.247647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:38.247852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:38.309066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 160/329] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:38.334891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f535510-7190-42eb-9375-f2cdf6839bba None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:38.344615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbabe9c-787a-465d-845a-a3b08740d090 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:38.385215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0855ccc-abc2-422e-9408-84a2c3e1013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:38.385984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855ccc-abc2-422e-9408-84a2c3e1013b None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:38.386404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855ccc-abc2-422e-9408-84a2c3e1013b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:38.386886 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0855ccc-abc2-422e-9408-84a2c3e1013b None admin] PATH_INFO: `/v3/system/users/b6fecb2bde1f4ea987c53ee2525c1ad7/roles/671315d9acb2489cae6944e5c4c5dc77` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:38.388416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0855ccc-abc2-422e-9408-84a2c3e1013b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b6fecb2bde1f4ea987c53ee2525c1ad7, role_id=671315d9acb2489cae6944e5c4c5dc77)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:38.389821 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0855ccc-abc2-422e-9408-84a2c3e1013b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:38.458229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f535510-7190-42eb-9375-f2cdf6839bba None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:38.459914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 170/330] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Aug 11 09:40:38 2022] PUT /identity/v3/system/users/b6fecb2bde1f4ea987c53ee2525c1ad7/roles/671315d9acb2489cae6944e5c4c5dc77 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 09:40:38.500542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e1a1a-d09c-4ee3-895a-da36a81ca471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:38.501596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e1a1a-d09c-4ee3-895a-da36a81ca471 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:38.502096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e1a1a-d09c-4ee3-895a-da36a81ca471 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:38.502575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e1a1a-d09c-4ee3-895a-da36a81ca471 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:38.515864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e1a1a-d09c-4ee3-895a-da36a81ca471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-267e1a1a-d09c-4ee3-895a-da36a81ca471 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:38.521091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:38.522088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:38.522859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:38.525636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 161/331] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:38.531483 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0855ccc-abc2-422e-9408-84a2c3e1013b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:38.633293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:38.654418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0855ccc-abc2-422e-9408-84a2c3e1013b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:38.696783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60033ea2-6266-477b-be9e-592c0369fb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:38.697609 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60033ea2-6266-477b-be9e-592c0369fb51 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:38.698194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60033ea2-6266-477b-be9e-592c0369fb51 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:38.698703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60033ea2-6266-477b-be9e-592c0369fb51 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:38.699810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60033ea2-6266-477b-be9e-592c0369fb51 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:38.700116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60033ea2-6266-477b-be9e-592c0369fb51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:38.701546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60033ea2-6266-477b-be9e-592c0369fb51 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:38.711399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 171/332] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:38.716827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e1a1a-d09c-4ee3-895a-da36a81ca471 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:38.734122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:38.833420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e1a1a-d09c-4ee3-895a-da36a81ca471 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:38.873663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b68ff-8e8d-496a-b7f5-0301944587d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:38.874307 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b68ff-8e8d-496a-b7f5-0301944587d2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:38.874586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b68ff-8e8d-496a-b7f5-0301944587d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:38.874860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b68ff-8e8d-496a-b7f5-0301944587d2 None admin] PATH_INFO: `/v3/users/f1cde60106ca432ebb921d4118184395` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:38.876017 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b68ff-8e8d-496a-b7f5-0301944587d2 None admin] RBAC: Authorizing `identity:get_user(user_id=f1cde60106ca432ebb921d4118184395)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:38.877510 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b68ff-8e8d-496a-b7f5-0301944587d2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:38.879485 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 162/333] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:38 2022] GET /identity/v3/users/f1cde60106ca432ebb921d4118184395 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:38.885579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60033ea2-6266-477b-be9e-592c0369fb51 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:39.004080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60033ea2-6266-477b-be9e-592c0369fb51 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:39.042695 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9301e6-4e62-464e-9df4-e081926bed19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:39.043314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9301e6-4e62-464e-9df4-e081926bed19 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:39.043598 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9301e6-4e62-464e-9df4-e081926bed19 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:39.043870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9301e6-4e62-464e-9df4-e081926bed19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:39.044608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9301e6-4e62-464e-9df4-e081926bed19 None admin] RBAC: Adding query filter params (role.id=bda5c75a326e4f5d9503960830898895, user.id=f1cde60106ca432ebb921d4118184395, scope.system=all) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:39.044919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9301e6-4e62-464e-9df4-e081926bed19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:39.046383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9301e6-4e62-464e-9df4-e081926bed19 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:39.055596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:39.059181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 172/334] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Aug 11 09:40:38 2022] GET /identity/v3/role_assignments?user.id=f1cde60106ca432ebb921d4118184395&scope.system=all&role.id=bda5c75a326e4f5d9503960830898895 => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:40.456732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cd046-ef1e-4543-9ac8-241e4da71bdb None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:40.457840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cd046-ef1e-4543-9ac8-241e4da71bdb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:40.458329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cd046-ef1e-4543-9ac8-241e4da71bdb None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:40.460338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 163/335] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:40.466204 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d14764-28ac-444e-a8d8-8824fabcc5e9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:40.467059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d14764-28ac-444e-a8d8-8824fabcc5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:40.467539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d14764-28ac-444e-a8d8-8824fabcc5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:40.491296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-76d14764-28ac-444e-a8d8-8824fabcc5e9 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:40.493057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-76d14764-28ac-444e-a8d8-8824fabcc5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:40.568988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 173/336] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:40.593175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387cd046-ef1e-4543-9ac8-241e4da71bdb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:40.688224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:40.713888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387cd046-ef1e-4543-9ac8-241e4da71bdb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:40.751990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647b69d-80e2-480b-9484-39852953143c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:40.752622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647b69d-80e2-480b-9484-39852953143c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:40.752895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647b69d-80e2-480b-9484-39852953143c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:40.753192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647b69d-80e2-480b-9484-39852953143c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:40.764071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647b69d-80e2-480b-9484-39852953143c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-5647b69d-80e2-480b-9484-39852953143c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:40.768767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:40.769660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:40.770428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 164/337] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:40.782247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d14764-28ac-444e-a8d8-8824fabcc5e9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:40.939888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d14764-28ac-444e-a8d8-8824fabcc5e9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:40.980080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:40.983830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f2b89-a54e-48a7-8db2-f9fb25ee5967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:40.984595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f2b89-a54e-48a7-8db2-f9fb25ee5967 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:40.984998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f2b89-a54e-48a7-8db2-f9fb25ee5967 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:40.985413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f2b89-a54e-48a7-8db2-f9fb25ee5967 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:40.986155 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f2b89-a54e-48a7-8db2-f9fb25ee5967 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:40.986469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f2b89-a54e-48a7-8db2-f9fb25ee5967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:40.987910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f2b89-a54e-48a7-8db2-f9fb25ee5967 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:41.000448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 174/338] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:41.006626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647b69d-80e2-480b-9484-39852953143c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:41.135594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647b69d-80e2-480b-9484-39852953143c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:41.184710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589e875-cace-4993-a30f-ee738723b328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:41.185806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589e875-cace-4993-a30f-ee738723b328 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:41.185806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589e875-cace-4993-a30f-ee738723b328 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:41.185965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589e875-cace-4993-a30f-ee738723b328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:41.201154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:41.252767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf9db9-d22e-4c74-841b-2294e5c28bed None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:41.253064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf9db9-d22e-4c74-841b-2294e5c28bed None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:41.253251 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf9db9-d22e-4c74-841b-2294e5c28bed None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:41.254829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 175/339] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:41.261497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b18f2f-1a65-4ec2-b98c-e532133e9e2d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:41.261777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b18f2f-1a65-4ec2-b98c-e532133e9e2d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:41.262053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b18f2f-1a65-4ec2-b98c-e532133e9e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:41.294611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589e875-cace-4993-a30f-ee738723b328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:41.295107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f3b18f2f-1a65-4ec2-b98c-e532133e9e2d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:41.296319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f3b18f2f-1a65-4ec2-b98c-e532133e9e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:41.296876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589e875-cace-4993-a30f-ee738723b328 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:41.385653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 176/340] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:41.387258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110096a-1703-4ad2-a4a9-d8213da65780 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:41.387767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110096a-1703-4ad2-a4a9-d8213da65780 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:41.388239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110096a-1703-4ad2-a4a9-d8213da65780 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:41.389697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 177/341] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:41.390897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f736d-d849-45a9-ba3e-0b9e7d43381a None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:41.391378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f736d-d849-45a9-ba3e-0b9e7d43381a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:41.391847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f736d-d849-45a9-ba3e-0b9e7d43381a None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:41.397339 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 178/342] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:41.402964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69b199-8a57-488b-b2f0-3e6b74692adc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:41.403510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69b199-8a57-488b-b2f0-3e6b74692adc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:41.403986 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69b199-8a57-488b-b2f0-3e6b74692adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:41.424096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5c69b199-8a57-488b-b2f0-3e6b74692adc None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:41.425671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5c69b199-8a57-488b-b2f0-3e6b74692adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:41.448979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 165/343] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:41.453506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006e65-7a31-43e3-b0a9-917c1c11a684 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:41.453506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006e65-7a31-43e3-b0a9-917c1c11a684 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:41.453506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006e65-7a31-43e3-b0a9-917c1c11a684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:41.471286 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5d006e65-7a31-43e3-b0a9-917c1c11a684 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:41.472551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5d006e65-7a31-43e3-b0a9-917c1c11a684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:41.512469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 179/344] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:41.513991 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69b199-8a57-488b-b2f0-3e6b74692adc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:41.549404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:41.562651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 166/345] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:41.563308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d006e65-7a31-43e3-b0a9-917c1c11a684 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:41.600059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:41.654609 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69b199-8a57-488b-b2f0-3e6b74692adc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:41.673803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d006e65-7a31-43e3-b0a9-917c1c11a684 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:41.694235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f410c8d-8359-41d4-b389-554179b765ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:41.695209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f410c8d-8359-41d4-b389-554179b765ef None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:41.695674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f410c8d-8359-41d4-b389-554179b765ef None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:41.696127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f410c8d-8359-41d4-b389-554179b765ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:41.708157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f410c8d-8359-41d4-b389-554179b765ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:41.712394 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab416583-e422-4e25-9264-37d1fd65d80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:41.713133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab416583-e422-4e25-9264-37d1fd65d80e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:41.713481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab416583-e422-4e25-9264-37d1fd65d80e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:41.713825 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab416583-e422-4e25-9264-37d1fd65d80e None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4f410c8d-8359-41d4-b389-554179b765ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:41.715315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:41.716168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:41.716919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:41.717557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab416583-e422-4e25-9264-37d1fd65d80e None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:41.717557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab416583-e422-4e25-9264-37d1fd65d80e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:41.719711 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 180/346] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:41.720769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f410c8d-8359-41d4-b389-554179b765ef None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:41.725654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 167/347] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:41 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:41.726224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab416583-e422-4e25-9264-37d1fd65d80e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:41.781105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:41.839955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab416583-e422-4e25-9264-37d1fd65d80e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:41.840624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f410c8d-8359-41d4-b389-554179b765ef None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:41.854852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:41.880543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032607f3-9885-49b7-b41b-9d3b33c8e20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:41.881608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032607f3-9885-49b7-b41b-9d3b33c8e20d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:41.881913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032607f3-9885-49b7-b41b-9d3b33c8e20d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:41.882248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032607f3-9885-49b7-b41b-9d3b33c8e20d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:41.883825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9dc7b-71dd-4d91-a7d7-49319ddcf32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:41.885330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9dc7b-71dd-4d91-a7d7-49319ddcf32e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:41.885823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9dc7b-71dd-4d91-a7d7-49319ddcf32e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:41.886298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9dc7b-71dd-4d91-a7d7-49319ddcf32e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:41.893855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032607f3-9885-49b7-b41b-9d3b33c8e20d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-032607f3-9885-49b7-b41b-9d3b33c8e20d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:41.899380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:41.900436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:41.901270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 168/348] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:41.901906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032607f3-9885-49b7-b41b-9d3b33c8e20d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:41.902811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9dc7b-71dd-4d91-a7d7-49319ddcf32e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-bde9dc7b-71dd-4d91-a7d7-49319ddcf32e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:41.909382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:41.910310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:41.911112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:41.914513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 181/349] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:41.921044 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9dc7b-71dd-4d91-a7d7-49319ddcf32e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:42.034911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:42.039517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032607f3-9885-49b7-b41b-9d3b33c8e20d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:42.044681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9dc7b-71dd-4d91-a7d7-49319ddcf32e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:42.076973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c6bdd-f325-4120-99a1-80d012b19f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:42.077790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c6bdd-f325-4120-99a1-80d012b19f4e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:42.078071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c6bdd-f325-4120-99a1-80d012b19f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:42.078344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c6bdd-f325-4120-99a1-80d012b19f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:42.079013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c6bdd-f325-4120-99a1-80d012b19f4e None admin] RBAC: Adding query filter params (name=member) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:42.079314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c6bdd-f325-4120-99a1-80d012b19f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:42.080691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c6bdd-f325-4120-99a1-80d012b19f4e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:42.085694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c96c-0acc-4ab9-a2e6-a66cd447a0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:42.086389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c96c-0acc-4ab9-a2e6-a66cd447a0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:42.086672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c96c-0acc-4ab9-a2e6-a66cd447a0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:42.086950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3c96c-0acc-4ab9-a2e6-a66cd447a0d8 None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:42.090459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 169/350] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:42.091296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c6bdd-f325-4120-99a1-80d012b19f4e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:42.095640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3c96c-0acc-4ab9-a2e6-a66cd447a0d8 None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:42.097184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3c96c-0acc-4ab9-a2e6-a66cd447a0d8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:42.099182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 182/351] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:41 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:42.100724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c96c-0acc-4ab9-a2e6-a66cd447a0d8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:42.153387 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:42.229297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3c96c-0acc-4ab9-a2e6-a66cd447a0d8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:42.232764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c6bdd-f325-4120-99a1-80d012b19f4e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:42.274133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5884-8567-4c82-a252-c2139314dce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:42.275104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5884-8567-4c82-a252-c2139314dce7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:42.275655 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5884-8567-4c82-a252-c2139314dce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:42.275981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a6ba1-1f10-4b01-96cf-d9a1887defa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:42.276377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5884-8567-4c82-a252-c2139314dce7 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:42.276603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a6ba1-1f10-4b01-96cf-d9a1887defa1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:42.276885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a6ba1-1f10-4b01-96cf-d9a1887defa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:42.277205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a6ba1-1f10-4b01-96cf-d9a1887defa1 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:42.278021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a6ba1-1f10-4b01-96cf-d9a1887defa1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:42.278328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a6ba1-1f10-4b01-96cf-d9a1887defa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:42.278650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b5884-8567-4c82-a252-c2139314dce7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:42.279141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b5884-8567-4c82-a252-c2139314dce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:42.279690 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a6ba1-1f10-4b01-96cf-d9a1887defa1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:42.280915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b5884-8567-4c82-a252-c2139314dce7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:42.290718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 183/352] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:42.291536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a6ba1-1f10-4b01-96cf-d9a1887defa1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:42.291747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 170/353] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:42.292742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5884-8567-4c82-a252-c2139314dce7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:42.328107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:42.417881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5884-8567-4c82-a252-c2139314dce7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:42.421467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:42.427645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a6ba1-1f10-4b01-96cf-d9a1887defa1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:42.459554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71b28c-1561-4945-9a19-e76577b557f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:42.460315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71b28c-1561-4945-9a19-e76577b557f8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:42.460676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71b28c-1561-4945-9a19-e76577b557f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:42.461038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71b28c-1561-4945-9a19-e76577b557f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:42.462011 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71b28c-1561-4945-9a19-e76577b557f8 None admin] RBAC: Adding query filter params (group.id=dbae207e1080407fa50c901b073e99fa, role.id=bda5c75a326e4f5d9503960830898895, scope.project.id=393ec3a506734c8b8c02823cb21848e2) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:42.462408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71b28c-1561-4945-9a19-e76577b557f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:42.464067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71b28c-1561-4945-9a19-e76577b557f8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:42.470620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 171/354] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Aug 11 09:40:42 2022] GET /identity/v3/role_assignments?group.id=dbae207e1080407fa50c901b073e99fa&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=bda5c75a326e4f5d9503960830898895 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:42.472742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71b28c-1561-4945-9a19-e76577b557f8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:42.473591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378e5bb-fd3a-425e-9e4e-c42e01e7daa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:42.474234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378e5bb-fd3a-425e-9e4e-c42e01e7daa8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:42.474517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378e5bb-fd3a-425e-9e4e-c42e01e7daa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:42.475349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378e5bb-fd3a-425e-9e4e-c42e01e7daa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:42.579313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378e5bb-fd3a-425e-9e4e-c42e01e7daa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:42.581323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378e5bb-fd3a-425e-9e4e-c42e01e7daa8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:42.591682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:42.639144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71b28c-1561-4945-9a19-e76577b557f8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:42.705626 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 184/355] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:42.706579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378e5bb-fd3a-425e-9e4e-c42e01e7daa8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:42.711393 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:42.711923 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46358114-98ee-4265-b211-7fae82a6324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:42.712592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46358114-98ee-4265-b211-7fae82a6324b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:42.712873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46358114-98ee-4265-b211-7fae82a6324b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:42.713186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46358114-98ee-4265-b211-7fae82a6324b None admin] PATH_INFO: `/v3/users/b6fecb2bde1f4ea987c53ee2525c1ad7` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:42.719526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46358114-98ee-4265-b211-7fae82a6324b None admin] RBAC: Authorizing `identity:get_user(user_id=b6fecb2bde1f4ea987c53ee2525c1ad7)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:42.721091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46358114-98ee-4265-b211-7fae82a6324b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:42.722780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 172/356] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:40:42 2022] GET /identity/v3/users/b6fecb2bde1f4ea987c53ee2525c1ad7 => generated 293 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:42.723551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46358114-98ee-4265-b211-7fae82a6324b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:42.844618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378e5bb-fd3a-425e-9e4e-c42e01e7daa8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:42.861435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46358114-98ee-4265-b211-7fae82a6324b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:42.889347 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db8dd14-fd29-440c-ae9e-60ca4ded0e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:42.890169 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8dd14-fd29-440c-ae9e-60ca4ded0e6b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:42.890455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8dd14-fd29-440c-ae9e-60ca4ded0e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:42.890732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db8dd14-fd29-440c-ae9e-60ca4ded0e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:42.905441 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678c324-002a-4f0e-bb14-ea8609611687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:42.906366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678c324-002a-4f0e-bb14-ea8609611687 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:42.906655 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678c324-002a-4f0e-bb14-ea8609611687 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:42.906932 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678c324-002a-4f0e-bb14-ea8609611687 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:42.908103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678c324-002a-4f0e-bb14-ea8609611687 None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:42.910354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678c324-002a-4f0e-bb14-ea8609611687 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:42.913273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 173/357] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:42 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:42.913970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678c324-002a-4f0e-bb14-ea8609611687 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:42.948052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:42.969412 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:42.976501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8dd14-fd29-440c-ae9e-60ca4ded0e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:42.978182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db8dd14-fd29-440c-ae9e-60ca4ded0e6b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:43.055605 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678c324-002a-4f0e-bb14-ea8609611687 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:43.103957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bd2ad-04a0-4f10-8cca-c55be6a52de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:43.104155 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 185/358] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:43.104489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bd2ad-04a0-4f10-8cca-c55be6a52de5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:43.104770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bd2ad-04a0-4f10-8cca-c55be6a52de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:43.105104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bd2ad-04a0-4f10-8cca-c55be6a52de5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:43.105355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778f467-695e-42bb-a0d5-648d3e424e35 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:43.105640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778f467-695e-42bb-a0d5-648d3e424e35 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:43.105829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bd2ad-04a0-4f10-8cca-c55be6a52de5 None admin] RBAC: Adding query filter params (role.id=671315d9acb2489cae6944e5c4c5dc77, user.id=b6fecb2bde1f4ea987c53ee2525c1ad7, scope.system=all) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:43.105920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778f467-695e-42bb-a0d5-648d3e424e35 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:43.106138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bd2ad-04a0-4f10-8cca-c55be6a52de5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:43.107260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 186/359] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:43.107561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bd2ad-04a0-4f10-8cca-c55be6a52de5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:43.107809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778f467-695e-42bb-a0d5-648d3e424e35 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:43.122826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 174/360] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Aug 11 09:40:42 2022] GET /identity/v3/role_assignments?user.id=b6fecb2bde1f4ea987c53ee2525c1ad7&scope.system=all&role.id=671315d9acb2489cae6944e5c4c5dc77 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:43.124103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e3ce8-90f2-436a-b4a9-8adef8339f2f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:43.124390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e3ce8-90f2-436a-b4a9-8adef8339f2f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:43.125833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e3ce8-90f2-436a-b4a9-8adef8339f2f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:43.127471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 175/361] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:43.128738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c5bb-7452-4d6f-9148-4d01b60168b8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:43.129030 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c5bb-7452-4d6f-9148-4d01b60168b8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:43.129353 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c5bb-7452-4d6f-9148-4d01b60168b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:43.150761 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-bc90c5bb-7452-4d6f-9148-4d01b60168b8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:43.152027 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bc90c5bb-7452-4d6f-9148-4d01b60168b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:43.234905 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 176/362] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:43.235736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90c5bb-7452-4d6f-9148-4d01b60168b8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:43.249534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:43.255999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:43.262182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778f467-695e-42bb-a0d5-648d3e424e35 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:43.309837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a3268-30ce-4677-91a5-942d0a6aaf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:43.310876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a3268-30ce-4677-91a5-942d0a6aaf5b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:43.311633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a3268-30ce-4677-91a5-942d0a6aaf5b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:43.312044 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a3268-30ce-4677-91a5-942d0a6aaf5b None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:43.313498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a3268-30ce-4677-91a5-942d0a6aaf5b None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:43.315724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a3268-30ce-4677-91a5-942d0a6aaf5b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:43.323236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 187/363] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:43 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:43.324493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b04d-dae4-48a7-b9fc-806292a7e178 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:43.324782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b04d-dae4-48a7-b9fc-806292a7e178 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:43.326848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b04d-dae4-48a7-b9fc-806292a7e178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:43.349609 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fa59b04d-dae4-48a7-b9fc-806292a7e178 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:43.351249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fa59b04d-dae4-48a7-b9fc-806292a7e178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:43.373420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90c5bb-7452-4d6f-9148-4d01b60168b8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:43.418194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643641b-42a8-4bec-88c6-841aa12a4c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:43.418926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643641b-42a8-4bec-88c6-841aa12a4c27 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:43.419210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643641b-42a8-4bec-88c6-841aa12a4c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:43.419479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643641b-42a8-4bec-88c6-841aa12a4c27 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:43.421709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643641b-42a8-4bec-88c6-841aa12a4c27 None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:43.423150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643641b-42a8-4bec-88c6-841aa12a4c27 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:43.424999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 177/364] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:43 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:43.425641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643641b-42a8-4bec-88c6-841aa12a4c27 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:43.441296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 188/365] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:43.442761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b04d-dae4-48a7-b9fc-806292a7e178 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:43.518339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:43.545518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643641b-42a8-4bec-88c6-841aa12a4c27 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:43.552537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:43.561500 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b04d-dae4-48a7-b9fc-806292a7e178 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:43.586944 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9c467-7c1a-47e4-ad18-b9b364b0bbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:43.587587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c467-7c1a-47e4-ad18-b9b364b0bbba None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:43.587867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c467-7c1a-47e4-ad18-b9b364b0bbba None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:43.588139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c467-7c1a-47e4-ad18-b9b364b0bbba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:43.603682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9c467-7c1a-47e4-ad18-b9b364b0bbba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:43.604347 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32287f-2fd7-433f-92a9-0c5a5ea80dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:43.605569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32287f-2fd7-433f-92a9-0c5a5ea80dcb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:43.606055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32287f-2fd7-433f-92a9-0c5a5ea80dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:43.606329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32287f-2fd7-433f-92a9-0c5a5ea80dcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:43.607064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32287f-2fd7-433f-92a9-0c5a5ea80dcb None admin] RBAC: Adding query filter params (group.id=dbae207e1080407fa50c901b073e99fa, role.id=bda5c75a326e4f5d9503960830898895, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:43.607372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32287f-2fd7-433f-92a9-0c5a5ea80dcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-fbb9c467-7c1a-47e4-ad18-b9b364b0bbba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:43.608462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:43.609374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:43.610641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:43.611234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32287f-2fd7-433f-92a9-0c5a5ea80dcb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:43.611300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 178/366] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:43.611300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9c467-7c1a-47e4-ad18-b9b364b0bbba None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:43.616374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 189/367] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Aug 11 09:40:43 2022] GET /identity/v3/role_assignments?group.id=dbae207e1080407fa50c901b073e99fa&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=bda5c75a326e4f5d9503960830898895 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:43.617231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32287f-2fd7-433f-92a9-0c5a5ea80dcb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:43.767026 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9c467-7c1a-47e4-ad18-b9b364b0bbba None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:43.769054 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32287f-2fd7-433f-92a9-0c5a5ea80dcb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:43.810862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a297c-ddb5-414f-9518-b6d9a1a8f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:43.811504 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a297c-ddb5-414f-9518-b6d9a1a8f177 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:43.811785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a297c-ddb5-414f-9518-b6d9a1a8f177 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:43.812072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a297c-ddb5-414f-9518-b6d9a1a8f177 None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:43.813449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a297c-ddb5-414f-9518-b6d9a1a8f177 None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:43.815010 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a297c-ddb5-414f-9518-b6d9a1a8f177 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:43.815221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319c9bd-37d7-4b6d-bcaa-b26f73533f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:43.815891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319c9bd-37d7-4b6d-bcaa-b26f73533f0e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:43.816177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319c9bd-37d7-4b6d-bcaa-b26f73533f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:43.816499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319c9bd-37d7-4b6d-bcaa-b26f73533f0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:43.817410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 179/368] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:43 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:43.819065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a297c-ddb5-414f-9518-b6d9a1a8f177 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:43.834932 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:43.835225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319c9bd-37d7-4b6d-bcaa-b26f73533f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d319c9bd-37d7-4b6d-bcaa-b26f73533f0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:43.840330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:43.841469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:43.842317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 190/369] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:43.848963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319c9bd-37d7-4b6d-bcaa-b26f73533f0e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:43.875632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:43.964774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a297c-ddb5-414f-9518-b6d9a1a8f177 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:43.970845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319c9bd-37d7-4b6d-bcaa-b26f73533f0e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:44.009417 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ae82c-d370-48e7-8ea9-2b060bd21903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:44.010513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ae82c-d370-48e7-8ea9-2b060bd21903 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:44.010931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ae82c-d370-48e7-8ea9-2b060bd21903 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:44.011518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ae82c-d370-48e7-8ea9-2b060bd21903 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:44.012326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ae82c-d370-48e7-8ea9-2b060bd21903 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:44.012616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94de10-06cd-435a-9049-9e32e8e98b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:44.012953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ae82c-d370-48e7-8ea9-2b060bd21903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:44.013275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94de10-06cd-435a-9049-9e32e8e98b4a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:44.013555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94de10-06cd-435a-9049-9e32e8e98b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:44.013831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94de10-06cd-435a-9049-9e32e8e98b4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:44.014565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94de10-06cd-435a-9049-9e32e8e98b4a None admin] RBAC: Adding query filter params (group.id=dbae207e1080407fa50c901b073e99fa, role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596, scope.project.id=393ec3a506734c8b8c02823cb21848e2) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:44.015175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ae82c-d370-48e7-8ea9-2b060bd21903 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:44.015905 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94de10-06cd-435a-9049-9e32e8e98b4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:44.017424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94de10-06cd-435a-9049-9e32e8e98b4a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:44.024219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 191/370] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Aug 11 09:40:43 2022] GET /identity/v3/role_assignments?group.id=dbae207e1080407fa50c901b073e99fa&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:44.026696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 180/371] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:44.029261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94de10-06cd-435a-9049-9e32e8e98b4a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:44.032490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ae82c-d370-48e7-8ea9-2b060bd21903 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:44.117470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:44.159775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ae82c-d370-48e7-8ea9-2b060bd21903 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:44.202879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c019e-67dc-4a2c-bb73-7c488115d898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:44.203514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c019e-67dc-4a2c-bb73-7c488115d898 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:44.203797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c019e-67dc-4a2c-bb73-7c488115d898 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:44.204074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c019e-67dc-4a2c-bb73-7c488115d898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:44.282298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:44.292691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c019e-67dc-4a2c-bb73-7c488115d898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:44.295203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c019e-67dc-4a2c-bb73-7c488115d898 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:44.305252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94de10-06cd-435a-9049-9e32e8e98b4a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:44.350504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853cef7-4228-467e-b9b0-927bda3d20c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:44.351122 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853cef7-4228-467e-b9b0-927bda3d20c8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:44.351402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853cef7-4228-467e-b9b0-927bda3d20c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:44.351675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853cef7-4228-467e-b9b0-927bda3d20c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:44.352351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853cef7-4228-467e-b9b0-927bda3d20c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:44.353642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853cef7-4228-467e-b9b0-927bda3d20c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:44.354701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853cef7-4228-467e-b9b0-927bda3d20c8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:44.366834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 192/372] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:44.373717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853cef7-4228-467e-b9b0-927bda3d20c8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:44.401436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:44.435473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 181/373] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:44.441050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c019e-67dc-4a2c-bb73-7c488115d898 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:44.507825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853cef7-4228-467e-b9b0-927bda3d20c8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:44.552031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafb29b-2cc7-44b5-9409-d92830c09475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:44.553275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafb29b-2cc7-44b5-9409-d92830c09475 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:44.553564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafb29b-2cc7-44b5-9409-d92830c09475 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:44.554165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafb29b-2cc7-44b5-9409-d92830c09475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:44.557455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c019e-67dc-4a2c-bb73-7c488115d898 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:44.573532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:44.599451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc34b4-5b80-4809-a7fb-d520297255ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:44.600689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc34b4-5b80-4809-a7fb-d520297255ef None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:44.601068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc34b4-5b80-4809-a7fb-d520297255ef None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:44.601576 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc34b4-5b80-4809-a7fb-d520297255ef None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:44.602817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc34b4-5b80-4809-a7fb-d520297255ef None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:44.604339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc34b4-5b80-4809-a7fb-d520297255ef None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:44.606432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 182/374] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:44 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:44.611854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc34b4-5b80-4809-a7fb-d520297255ef None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:44.639677 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aafb29b-2cc7-44b5-9409-d92830c09475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:44.641374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aafb29b-2cc7-44b5-9409-d92830c09475 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:44.721387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:44.757479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 193/375] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:44.770166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafb29b-2cc7-44b5-9409-d92830c09475 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:44.839431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc34b4-5b80-4809-a7fb-d520297255ef None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:44.870548 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:44.882364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45d489-d226-4ff1-a6d2-d1961f1b39f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:44.883398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45d489-d226-4ff1-a6d2-d1961f1b39f3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:44.883855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45d489-d226-4ff1-a6d2-d1961f1b39f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:44.884785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45d489-d226-4ff1-a6d2-d1961f1b39f3 None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:44.886311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45d489-d226-4ff1-a6d2-d1961f1b39f3 None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:44.888251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45d489-d226-4ff1-a6d2-d1961f1b39f3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:44.890613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 183/376] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:44 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:44.901103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45d489-d226-4ff1-a6d2-d1961f1b39f3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:44.961049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafb29b-2cc7-44b5-9409-d92830c09475 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:45.006979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b482e07-cff2-4ad4-8009-8ca34af25e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:45.007628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b482e07-cff2-4ad4-8009-8ca34af25e4e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:45.007908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b482e07-cff2-4ad4-8009-8ca34af25e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:45.008181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b482e07-cff2-4ad4-8009-8ca34af25e4e None admin] PATH_INFO: `/v3/projects/c055f43eb0f74f81a1a2e702ea1b7964` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:45.009367 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b482e07-cff2-4ad4-8009-8ca34af25e4e None admin] RBAC: Authorizing `identity:get_project(project_id=c055f43eb0f74f81a1a2e702ea1b7964)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:45.010868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b482e07-cff2-4ad4-8009-8ca34af25e4e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:45.012885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 194/377] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:44 2022] GET /identity/v3/projects/c055f43eb0f74f81a1a2e702ea1b7964 => generated 344 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:45.014032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f17b7-89a6-4899-b02d-4357dcdc5b85 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:45.014318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f17b7-89a6-4899-b02d-4357dcdc5b85 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:45.014593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f17b7-89a6-4899-b02d-4357dcdc5b85 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:45.015783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 195/378] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:45.021228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235853d-378d-498f-ac00-979a8c386751 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:45.021590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235853d-378d-498f-ac00-979a8c386751 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:45.022100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235853d-378d-498f-ac00-979a8c386751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:45.043053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0235853d-378d-498f-ac00-979a8c386751 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:45.044018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45d489-d226-4ff1-a6d2-d1961f1b39f3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:45.044307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0235853d-378d-498f-ac00-979a8c386751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:45.046654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:45.109198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d895b80-b8b7-47ea-b38c-1b1b09a046f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:45.109752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d895b80-b8b7-47ea-b38c-1b1b09a046f8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:45.110186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d895b80-b8b7-47ea-b38c-1b1b09a046f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:45.110566 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d895b80-b8b7-47ea-b38c-1b1b09a046f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:45.111999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d895b80-b8b7-47ea-b38c-1b1b09a046f8 None admin] RBAC: Adding query filter params (group.id=dbae207e1080407fa50c901b073e99fa, role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:45.112587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d895b80-b8b7-47ea-b38c-1b1b09a046f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:45.114644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d895b80-b8b7-47ea-b38c-1b1b09a046f8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:45.123796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 184/379] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Aug 11 09:40:44 2022] GET /identity/v3/role_assignments?group.id=dbae207e1080407fa50c901b073e99fa&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:45.124145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d895b80-b8b7-47ea-b38c-1b1b09a046f8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:45.155240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 196/380] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:45.185965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235853d-378d-498f-ac00-979a8c386751 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:45.273604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d895b80-b8b7-47ea-b38c-1b1b09a046f8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:45.312145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:45.351549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe5a16-269e-455b-8693-2b0527a5a32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:45.352196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe5a16-269e-455b-8693-2b0527a5a32f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:45.352474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe5a16-269e-455b-8693-2b0527a5a32f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:45.352745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe5a16-269e-455b-8693-2b0527a5a32f None admin] PATH_INFO: `/v3/groups/1378a8a95dd949649fc2c9d49b45133e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:45.363281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe5a16-269e-455b-8693-2b0527a5a32f None admin] RBAC: Authorizing `identity:get_group(group_id=1378a8a95dd949649fc2c9d49b45133e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:45.364697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe5a16-269e-455b-8693-2b0527a5a32f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:45.370148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 185/381] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:45 2022] GET /identity/v3/groups/1378a8a95dd949649fc2c9d49b45133e => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:45.376008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe5a16-269e-455b-8693-2b0527a5a32f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:45.402218 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235853d-378d-498f-ac00-979a8c386751 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:45.440318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:45.451684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc77fb6-9f7b-4416-813b-40172a56a7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:45.452328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc77fb6-9f7b-4416-813b-40172a56a7aa None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:45.452610 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc77fb6-9f7b-4416-813b-40172a56a7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:45.452882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc77fb6-9f7b-4416-813b-40172a56a7aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:45.465594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc77fb6-9f7b-4416-813b-40172a56a7aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-adc77fb6-9f7b-4416-813b-40172a56a7aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:45.470910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:45.471787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:45.472854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 197/382] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:45.478546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc77fb6-9f7b-4416-813b-40172a56a7aa None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:45.608258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc77fb6-9f7b-4416-813b-40172a56a7aa None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:45.615662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:45.623308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe5a16-269e-455b-8693-2b0527a5a32f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:45.680345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8fdb2-8148-47e8-a45a-d3c220246875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:45.681603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8fdb2-8148-47e8-a45a-d3c220246875 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:45.682066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8fdb2-8148-47e8-a45a-d3c220246875 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:45.682518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8fdb2-8148-47e8-a45a-d3c220246875 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:45.683401 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8fdb2-8148-47e8-a45a-d3c220246875 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:45.684374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8fdb2-8148-47e8-a45a-d3c220246875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:45.686313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8fdb2-8148-47e8-a45a-d3c220246875 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:45.692942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff20936-394e-4465-8a59-ec75756f17e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:45.694643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff20936-394e-4465-8a59-ec75756f17e3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:45.695213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff20936-394e-4465-8a59-ec75756f17e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:45.695665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff20936-394e-4465-8a59-ec75756f17e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:45.696582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff20936-394e-4465-8a59-ec75756f17e3 None admin] RBAC: Adding query filter params (group.id=1378a8a95dd949649fc2c9d49b45133e, role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=c055f43eb0f74f81a1a2e702ea1b7964) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:45.699049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff20936-394e-4465-8a59-ec75756f17e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:45.700680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff20936-394e-4465-8a59-ec75756f17e3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:45.710920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 198/383] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:45.714049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 186/384] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Aug 11 09:40:45 2022] GET /identity/v3/role_assignments?group.id=1378a8a95dd949649fc2c9d49b45133e&scope.project.id=c055f43eb0f74f81a1a2e702ea1b7964&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 260 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:45.725922 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8fdb2-8148-47e8-a45a-d3c220246875 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:45.878538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8fdb2-8148-47e8-a45a-d3c220246875 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:45.944973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3af7a-5198-48e4-9983-6ff61f3c88a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:45.945987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3af7a-5198-48e4-9983-6ff61f3c88a9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:45.946454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3af7a-5198-48e4-9983-6ff61f3c88a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:45.946908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3af7a-5198-48e4-9983-6ff61f3c88a9 None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:45.948217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3af7a-5198-48e4-9983-6ff61f3c88a9 None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:45.949849 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3af7a-5198-48e4-9983-6ff61f3c88a9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:45.951963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 199/385] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:45 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:45.957083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff20936-394e-4465-8a59-ec75756f17e3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:46.073574 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:46.080761 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff20936-394e-4465-8a59-ec75756f17e3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:46.121507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee23c07-c5c1-44f1-a68d-63a0833a456e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:46.122656 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee23c07-c5c1-44f1-a68d-63a0833a456e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:46.123475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee23c07-c5c1-44f1-a68d-63a0833a456e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:46.123751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee23c07-c5c1-44f1-a68d-63a0833a456e None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:46.124920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee23c07-c5c1-44f1-a68d-63a0833a456e None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:46.126387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee23c07-c5c1-44f1-a68d-63a0833a456e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:46.128361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 187/386] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:45 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:46.134348 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3af7a-5198-48e4-9983-6ff61f3c88a9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:46.197394 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:46.292253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3af7a-5198-48e4-9983-6ff61f3c88a9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:46.337262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f503af-6286-4f1c-9053-f1155240b8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:46.337938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f503af-6286-4f1c-9053-f1155240b8b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:46.338224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f503af-6286-4f1c-9053-f1155240b8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:46.338502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f503af-6286-4f1c-9053-f1155240b8b9 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2/groups/dbae207e1080407fa50c901b073e99fa/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:46.342260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f503af-6286-4f1c-9053-f1155240b8b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=393ec3a506734c8b8c02823cb21848e2, group_id=dbae207e1080407fa50c901b073e99fa, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:46.343766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f503af-6286-4f1c-9053-f1155240b8b9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:46.420115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 200/387] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Aug 11 09:40:46 2022] PUT /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2/groups/dbae207e1080407fa50c901b073e99fa/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:46.650715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a315c-f76f-40d7-b1ab-14bc6275e455 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:46.651210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a315c-f76f-40d7-b1ab-14bc6275e455 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:46.651282 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a315c-f76f-40d7-b1ab-14bc6275e455 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:46.652689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 188/388] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:46.658541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f858c7-746a-4481-8077-0229b27b88ab None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:46.658828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f858c7-746a-4481-8077-0229b27b88ab None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:46.659105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f858c7-746a-4481-8077-0229b27b88ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:46.681148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c3f858c7-746a-4481-8077-0229b27b88ab None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:46.682271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c3f858c7-746a-4481-8077-0229b27b88ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:46.762856 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 201/389] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:46.775836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d1d8b-c9d3-40cb-89f8-f5355b862179 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:46.776131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d1d8b-c9d3-40cb-89f8-f5355b862179 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:46.776451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d1d8b-c9d3-40cb-89f8-f5355b862179 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:46.777838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 189/390] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:46.783271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44417fe-cf13-49ac-a249-5308067a4661 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:46.783782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44417fe-cf13-49ac-a249-5308067a4661 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:46.784272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44417fe-cf13-49ac-a249-5308067a4661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:46.806322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d1d8b-c9d3-40cb-89f8-f5355b862179 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:46.811205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c44417fe-cf13-49ac-a249-5308067a4661 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:46.817515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c44417fe-cf13-49ac-a249-5308067a4661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:46.855200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:46.954968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 202/391] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:46.984224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d1d8b-c9d3-40cb-89f8-f5355b862179 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:47.006113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44417fe-cf13-49ac-a249-5308067a4661 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:47.054619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e6229-b70e-49ab-8152-0ffd3cf5ed11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:47.055354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e6229-b70e-49ab-8152-0ffd3cf5ed11 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:47.055640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e6229-b70e-49ab-8152-0ffd3cf5ed11 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:47.055909 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e6229-b70e-49ab-8152-0ffd3cf5ed11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:47.080647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e6229-b70e-49ab-8152-0ffd3cf5ed11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-353e6229-b70e-49ab-8152-0ffd3cf5ed11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:47.088256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:47.089176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:47.089953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:47.095306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 190/392] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:47.100370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e6229-b70e-49ab-8152-0ffd3cf5ed11 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:47.151956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:47.199162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44417fe-cf13-49ac-a249-5308067a4661 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:47.270864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b295f5a-8437-4ea2-b0b7-50ca3f6db663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:47.271672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b295f5a-8437-4ea2-b0b7-50ca3f6db663 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:47.272324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b295f5a-8437-4ea2-b0b7-50ca3f6db663 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:47.272638 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b295f5a-8437-4ea2-b0b7-50ca3f6db663 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:47.274855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e6229-b70e-49ab-8152-0ffd3cf5ed11 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:47.284913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b295f5a-8437-4ea2-b0b7-50ca3f6db663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-3b295f5a-8437-4ea2-b0b7-50ca3f6db663 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:47.292099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:47.292977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:47.293783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:47.296350 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 203/393] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:47.300012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b295f5a-8437-4ea2-b0b7-50ca3f6db663 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:47.311039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79837c65-9270-4911-9ff8-10c0fffe4e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:47.311665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79837c65-9270-4911-9ff8-10c0fffe4e6b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:47.311943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79837c65-9270-4911-9ff8-10c0fffe4e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:47.312219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79837c65-9270-4911-9ff8-10c0fffe4e6b None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:47.313000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79837c65-9270-4911-9ff8-10c0fffe4e6b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:47.314493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79837c65-9270-4911-9ff8-10c0fffe4e6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:47.315886 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79837c65-9270-4911-9ff8-10c0fffe4e6b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:47.320148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:47.327999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 191/394] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:47.334285 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79837c65-9270-4911-9ff8-10c0fffe4e6b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:47.449425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:47.454846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79837c65-9270-4911-9ff8-10c0fffe4e6b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:47.460427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b295f5a-8437-4ea2-b0b7-50ca3f6db663 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:47.504521 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6220c73-808d-4cb3-a8b7-911613a3c2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:47.505187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220c73-808d-4cb3-a8b7-911613a3c2ed None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:47.505474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220c73-808d-4cb3-a8b7-911613a3c2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:47.506289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220c73-808d-4cb3-a8b7-911613a3c2ed None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:47.507386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6220c73-808d-4cb3-a8b7-911613a3c2ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:47.507701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6220c73-808d-4cb3-a8b7-911613a3c2ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:47.509113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6220c73-808d-4cb3-a8b7-911613a3c2ed None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:47.520149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eea15-c220-4752-9d09-cd6548939591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:47.520786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eea15-c220-4752-9d09-cd6548939591 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:47.521211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eea15-c220-4752-9d09-cd6548939591 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:47.521543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eea15-c220-4752-9d09-cd6548939591 None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:47.523747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eea15-c220-4752-9d09-cd6548939591 None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:47.525408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eea15-c220-4752-9d09-cd6548939591 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:47.526985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 192/395] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:47 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:47.529128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 204/396] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:47.534497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eea15-c220-4752-9d09-cd6548939591 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:47.536039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6220c73-808d-4cb3-a8b7-911613a3c2ed None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:47.674632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:47.704994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6220c73-808d-4cb3-a8b7-911613a3c2ed None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:47.710865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eea15-c220-4752-9d09-cd6548939591 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:47.750577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02312bf-f587-4245-bd6d-ede08c696a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:47.751213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02312bf-f587-4245-bd6d-ede08c696a9d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:47.751481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02312bf-f587-4245-bd6d-ede08c696a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:47.751756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02312bf-f587-4245-bd6d-ede08c696a9d None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:47.752862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02312bf-f587-4245-bd6d-ede08c696a9d None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:47.753106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d016e40-b281-4d10-8926-6c3c78a48de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:47.753971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d016e40-b281-4d10-8926-6c3c78a48de2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:47.754325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02312bf-f587-4245-bd6d-ede08c696a9d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:47.754663 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d016e40-b281-4d10-8926-6c3c78a48de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:47.755126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d016e40-b281-4d10-8926-6c3c78a48de2 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:47.756226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 193/397] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:47 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:47.757078 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d016e40-b281-4d10-8926-6c3c78a48de2 None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:47.757201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876da73-1cf7-4553-a8a3-eb6ab71c2136 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:47.757489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876da73-1cf7-4553-a8a3-eb6ab71c2136 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:47.757762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876da73-1cf7-4553-a8a3-eb6ab71c2136 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:47.759189 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 194/398] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:47.759445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d016e40-b281-4d10-8926-6c3c78a48de2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:47.761539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876da73-1cf7-4553-a8a3-eb6ab71c2136 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:47.771039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 205/399] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:47 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:47.771039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2a641-0915-4b58-bf97-4b5f2f574638 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:47.771039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2a641-0915-4b58-bf97-4b5f2f574638 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:47.771039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2a641-0915-4b58-bf97-4b5f2f574638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:47.791597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9db2a641-0915-4b58-bf97-4b5f2f574638 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:47.792850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9db2a641-0915-4b58-bf97-4b5f2f574638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:47.797463 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:47.898298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 206/400] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:47.899175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2a641-0915-4b58-bf97-4b5f2f574638 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:47.913416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876da73-1cf7-4553-a8a3-eb6ab71c2136 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:47.957602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419acb3-9d73-4e43-99ad-648086e0b158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:47.958576 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419acb3-9d73-4e43-99ad-648086e0b158 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:47.959194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419acb3-9d73-4e43-99ad-648086e0b158 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:47.959699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419acb3-9d73-4e43-99ad-648086e0b158 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:47.961067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419acb3-9d73-4e43-99ad-648086e0b158 None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:47.963256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419acb3-9d73-4e43-99ad-648086e0b158 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:47.966107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 195/401] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:47 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:47.967130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419acb3-9d73-4e43-99ad-648086e0b158 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:48.017370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:48.048802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2a641-0915-4b58-bf97-4b5f2f574638 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:48.081644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:48.097004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419acb3-9d73-4e43-99ad-648086e0b158 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:48.107706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092933c8-6c8c-4dde-ac94-e2754c8287c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:48.109801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092933c8-6c8c-4dde-ac94-e2754c8287c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:48.110316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092933c8-6c8c-4dde-ac94-e2754c8287c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:48.111592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092933c8-6c8c-4dde-ac94-e2754c8287c2 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2/groups/dbae207e1080407fa50c901b073e99fa/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:48.115979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092933c8-6c8c-4dde-ac94-e2754c8287c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=393ec3a506734c8b8c02823cb21848e2, group_id=dbae207e1080407fa50c901b073e99fa, role_id=cfdf14ad1f2e4a7d9eeed35bc4ffa596)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:48.118301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092933c8-6c8c-4dde-ac94-e2754c8287c2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:48.144372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97211d4-ffba-46fb-a0f3-4f388ed29b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:48.145104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97211d4-ffba-46fb-a0f3-4f388ed29b0c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:48.145384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97211d4-ffba-46fb-a0f3-4f388ed29b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:48.145715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97211d4-ffba-46fb-a0f3-4f388ed29b0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:48.157054 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97211d4-ffba-46fb-a0f3-4f388ed29b0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a97211d4-ffba-46fb-a0f3-4f388ed29b0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:48.163578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:48.164518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:48.165312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:48.167520 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 196/402] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:48.167520 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97211d4-ffba-46fb-a0f3-4f388ed29b0c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:48.207951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 207/403] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Aug 11 09:40:47 2022] PUT /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2/groups/dbae207e1080407fa50c901b073e99fa/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:48.218152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092933c8-6c8c-4dde-ac94-e2754c8287c2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:48.284491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97211d4-ffba-46fb-a0f3-4f388ed29b0c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:48.323368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cbfa3-41cd-4e6d-8f3c-8282c28e4f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:48.324356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbfa3-41cd-4e6d-8f3c-8282c28e4f8e None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:48.324854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbfa3-41cd-4e6d-8f3c-8282c28e4f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:48.325357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cbfa3-41cd-4e6d-8f3c-8282c28e4f8e None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/groups/dbae207e1080407fa50c901b073e99fa/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:48.330814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cbfa3-41cd-4e6d-8f3c-8282c28e4f8e None admin] RBAC: Authorizing `identity:create_grant(project_id=2fb27f0238c0484ca533d76cafdcc6d6, group_id=dbae207e1080407fa50c901b073e99fa, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:48.332540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cbfa3-41cd-4e6d-8f3c-8282c28e4f8e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:48.394913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092933c8-6c8c-4dde-ac94-e2754c8287c2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:48.411829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 197/404] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Aug 11 09:40:48 2022] PUT /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/groups/dbae207e1080407fa50c901b073e99fa/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:48.453500 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69a6a3-8c2e-4d95-868d-c19c23ea6752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:48.454489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69a6a3-8c2e-4d95-868d-c19c23ea6752 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:48.454994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69a6a3-8c2e-4d95-868d-c19c23ea6752 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:48.455437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69a6a3-8c2e-4d95-868d-c19c23ea6752 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:48.456301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69a6a3-8c2e-4d95-868d-c19c23ea6752 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:48.456847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69a6a3-8c2e-4d95-868d-c19c23ea6752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:48.458479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69a6a3-8c2e-4d95-868d-c19c23ea6752 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:48.461373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:48.472443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 208/405] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:48.481081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cbfa3-41cd-4e6d-8f3c-8282c28e4f8e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:48.497478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:48.551719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42df94-c152-4bee-ada6-f05e2409c3df None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:48.552063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42df94-c152-4bee-ada6-f05e2409c3df None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:48.552272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42df94-c152-4bee-ada6-f05e2409c3df None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:48.554720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 209/406] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:48.580205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5d48b-6da3-45d4-ae9b-fbe2c198bac6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:48.580502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5d48b-6da3-45d4-ae9b-fbe2c198bac6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:48.581783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5d48b-6da3-45d4-ae9b-fbe2c198bac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:48.609204 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cbfa3-41cd-4e6d-8f3c-8282c28e4f8e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:48.614249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-77c5d48b-6da3-45d4-ae9b-fbe2c198bac6 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:48.616114 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-77c5d48b-6da3-45d4-ae9b-fbe2c198bac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:48.656460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a89b5-b2e1-4471-b64b-cb5146ed19e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:48.657133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a89b5-b2e1-4471-b64b-cb5146ed19e5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:48.657411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a89b5-b2e1-4471-b64b-cb5146ed19e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:48.657685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a89b5-b2e1-4471-b64b-cb5146ed19e5 None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:48.658957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a89b5-b2e1-4471-b64b-cb5146ed19e5 None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:48.661156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a89b5-b2e1-4471-b64b-cb5146ed19e5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:48.663997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 198/407] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:48 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:48.672028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a89b5-b2e1-4471-b64b-cb5146ed19e5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:48.711689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 210/408] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:48.743394 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5d48b-6da3-45d4-ae9b-fbe2c198bac6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:48.761857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:48.789087 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a89b5-b2e1-4471-b64b-cb5146ed19e5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:48.833245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c440f-f94a-4c51-9572-b80611189ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:48.834216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c440f-f94a-4c51-9572-b80611189ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:48.834498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c440f-f94a-4c51-9572-b80611189ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:48.834770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c440f-f94a-4c51-9572-b80611189ee5 None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:48.839083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c440f-f94a-4c51-9572-b80611189ee5 None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:48.840502 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c440f-f94a-4c51-9572-b80611189ee5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:48.850445 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 199/409] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:48 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:48.851473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c440f-f94a-4c51-9572-b80611189ee5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:48.903549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5d48b-6da3-45d4-ae9b-fbe2c198bac6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:48.941235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:48.951627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276dfaf-31fa-4457-ba07-8611b467e83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:48.952430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276dfaf-31fa-4457-ba07-8611b467e83d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:48.952717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276dfaf-31fa-4457-ba07-8611b467e83d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:48.952990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276dfaf-31fa-4457-ba07-8611b467e83d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:48.964222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276dfaf-31fa-4457-ba07-8611b467e83d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f276dfaf-31fa-4457-ba07-8611b467e83d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:48.969448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:48.970319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:48.971334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 211/410] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:40:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:48.973343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701e2d6-0669-491e-97f4-5e5888bddd9a None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:48.973343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701e2d6-0669-491e-97f4-5e5888bddd9a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:48.973343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701e2d6-0669-491e-97f4-5e5888bddd9a None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:48.975383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 212/411] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:48.979910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701e2d6-0669-491e-97f4-5e5888bddd9a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:48.999602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c440f-f94a-4c51-9572-b80611189ee5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:49.039650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e6a77-b493-4ba8-9a6e-8147efe1130f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:49.040367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e6a77-b493-4ba8-9a6e-8147efe1130f None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:49.040647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e6a77-b493-4ba8-9a6e-8147efe1130f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:49.040918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e6a77-b493-4ba8-9a6e-8147efe1130f None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/groups/dbae207e1080407fa50c901b073e99fa/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:49.043400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e6a77-b493-4ba8-9a6e-8147efe1130f None admin] RBAC: Authorizing `identity:create_grant(project_id=2fb27f0238c0484ca533d76cafdcc6d6, group_id=dbae207e1080407fa50c901b073e99fa, role_id=cfdf14ad1f2e4a7d9eeed35bc4ffa596)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:49.044887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e6a77-b493-4ba8-9a6e-8147efe1130f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:49.096457 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701e2d6-0669-491e-97f4-5e5888bddd9a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:49.121407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 200/412] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Aug 11 09:40:48 2022] PUT /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6/groups/dbae207e1080407fa50c901b073e99fa/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:49.122873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5c9b9-4c53-4feb-8bda-7c55d1100439 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:49.123159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5c9b9-4c53-4feb-8bda-7c55d1100439 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:49.123434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5c9b9-4c53-4feb-8bda-7c55d1100439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:49.139588 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac87da0-b55d-495c-b983-62273b566e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:49.140585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac87da0-b55d-495c-b983-62273b566e32 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:49.141139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac87da0-b55d-495c-b983-62273b566e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:49.141627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac87da0-b55d-495c-b983-62273b566e32 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:49.142529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac87da0-b55d-495c-b983-62273b566e32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:49.143081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac87da0-b55d-495c-b983-62273b566e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:49.144735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac87da0-b55d-495c-b983-62273b566e32 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:49.152661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:49.156990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 213/413] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:49.163764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-71e5c9b9-4c53-4feb-8bda-7c55d1100439 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:49.164040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac87da0-b55d-495c-b983-62273b566e32 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:49.165042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-71e5c9b9-4c53-4feb-8bda-7c55d1100439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:49.231438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:49.290220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac87da0-b55d-495c-b983-62273b566e32 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:49.330434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 201/414] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:49.331355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93667742-723c-4e11-ac22-f38dbf935500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:49.332245 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93667742-723c-4e11-ac22-f38dbf935500 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:49.332703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93667742-723c-4e11-ac22-f38dbf935500 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:49.333191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93667742-723c-4e11-ac22-f38dbf935500 None admin] PATH_INFO: `/v3/groups/1378a8a95dd949649fc2c9d49b45133e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:49.335112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93667742-723c-4e11-ac22-f38dbf935500 None admin] RBAC: Authorizing `identity:get_group(group_id=1378a8a95dd949649fc2c9d49b45133e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:49.336731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93667742-723c-4e11-ac22-f38dbf935500 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:49.338757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 214/415] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:49 2022] GET /identity/v3/groups/1378a8a95dd949649fc2c9d49b45133e => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:49.349125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5c9b9-4c53-4feb-8bda-7c55d1100439 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:49.361636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93667742-723c-4e11-ac22-f38dbf935500 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:49.492777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5c9b9-4c53-4feb-8bda-7c55d1100439 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:49.495804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93667742-723c-4e11-ac22-f38dbf935500 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:49.528686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:49.559881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688d88a-13c1-46b4-bbf8-8777d5b40342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:49.561803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688d88a-13c1-46b4-bbf8-8777d5b40342 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:49.563165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688d88a-13c1-46b4-bbf8-8777d5b40342 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:49.563639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688d88a-13c1-46b4-bbf8-8777d5b40342 None admin] PATH_INFO: `/v3/projects/c055f43eb0f74f81a1a2e702ea1b7964` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:49.564437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d7855-cdc1-43f9-886b-836a6c8f7d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:49.565857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d7855-cdc1-43f9-886b-836a6c8f7d84 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:49.566661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688d88a-13c1-46b4-bbf8-8777d5b40342 None admin] RBAC: Authorizing `identity:get_project(project_id=c055f43eb0f74f81a1a2e702ea1b7964)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:49.567423 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d7855-cdc1-43f9-886b-836a6c8f7d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:49.569982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688d88a-13c1-46b4-bbf8-8777d5b40342 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:49.570675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d7855-cdc1-43f9-886b-836a6c8f7d84 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:49.578080 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 202/416] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:49 2022] GET /identity/v3/projects/c055f43eb0f74f81a1a2e702ea1b7964 => generated 344 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:49.581365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688d88a-13c1-46b4-bbf8-8777d5b40342 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:49.588935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d7855-cdc1-43f9-886b-836a6c8f7d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:49.589345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-db7d7855-cdc1-43f9-886b-836a6c8f7d84 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:49.598831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:49.603321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:49.604113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 215/417] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:49.607842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d7855-cdc1-43f9-886b-836a6c8f7d84 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:49.714917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d7855-cdc1-43f9-886b-836a6c8f7d84 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:49.750934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897cfde-db1f-4524-ac40-069ec5d9581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:49.751575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897cfde-db1f-4524-ac40-069ec5d9581b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:49.751856 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897cfde-db1f-4524-ac40-069ec5d9581b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:49.752132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897cfde-db1f-4524-ac40-069ec5d9581b None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:49.752824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897cfde-db1f-4524-ac40-069ec5d9581b None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:49.753161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897cfde-db1f-4524-ac40-069ec5d9581b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:49.757041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897cfde-db1f-4524-ac40-069ec5d9581b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:49.765081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 216/418] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:49.778200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897cfde-db1f-4524-ac40-069ec5d9581b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:49.825404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:49.846214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688d88a-13c1-46b4-bbf8-8777d5b40342 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:49.910592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da5dfb-f2e1-422b-9b69-9088a9342ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:49.911170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da5dfb-f2e1-422b-9b69-9088a9342ed2 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:49.911450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da5dfb-f2e1-422b-9b69-9088a9342ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:49.911825 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da5dfb-f2e1-422b-9b69-9088a9342ed2 None admin] PATH_INFO: `/v3/projects/c055f43eb0f74f81a1a2e702ea1b7964/groups/1378a8a95dd949649fc2c9d49b45133e/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:49.915219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da5dfb-f2e1-422b-9b69-9088a9342ed2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c055f43eb0f74f81a1a2e702ea1b7964, group_id=1378a8a95dd949649fc2c9d49b45133e, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:49.917862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897cfde-db1f-4524-ac40-069ec5d9581b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:49.920185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da5dfb-f2e1-422b-9b69-9088a9342ed2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:49.957428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1d5ba-c942-4e38-83f7-bc8d19811304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:49.958045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1d5ba-c942-4e38-83f7-bc8d19811304 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:49.958327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1d5ba-c942-4e38-83f7-bc8d19811304 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:49.959168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1d5ba-c942-4e38-83f7-bc8d19811304 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:49.995611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 203/419] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Aug 11 09:40:49 2022] PUT /identity/v3/projects/c055f43eb0f74f81a1a2e702ea1b7964/groups/1378a8a95dd949649fc2c9d49b45133e/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:40:50.046471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1d5ba-c942-4e38-83f7-bc8d19811304 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:50.048136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1d5ba-c942-4e38-83f7-bc8d19811304 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:50.103346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:50.186051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 217/420] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 09:40:50.186155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da5dfb-f2e1-422b-9b69-9088a9342ed2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:50.260369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:50.300707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da5dfb-f2e1-422b-9b69-9088a9342ed2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:50.337987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7c3a5-3ce2-4288-91e1-404ed7590d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:50.338697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7c3a5-3ce2-4288-91e1-404ed7590d26 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:50.338979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7c3a5-3ce2-4288-91e1-404ed7590d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:50.339254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7c3a5-3ce2-4288-91e1-404ed7590d26 None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:50.340429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7c3a5-3ce2-4288-91e1-404ed7590d26 None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:50.341900 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7c3a5-3ce2-4288-91e1-404ed7590d26 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:50.343838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 204/421] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:50 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:50.350096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1d5ba-c942-4e38-83f7-bc8d19811304 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:50.486684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1d5ba-c942-4e38-83f7-bc8d19811304 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:50.529958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128f789-b14c-4db2-828c-363ab5818484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:50.530603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128f789-b14c-4db2-828c-363ab5818484 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:50.530886 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128f789-b14c-4db2-828c-363ab5818484 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:50.531219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128f789-b14c-4db2-828c-363ab5818484 None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:50.532350 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128f789-b14c-4db2-828c-363ab5818484 None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:50.533802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128f789-b14c-4db2-828c-363ab5818484 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:50.535612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 218/422] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:50 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:50.541396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7c3a5-3ce2-4288-91e1-404ed7590d26 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:50.649227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7c3a5-3ce2-4288-91e1-404ed7590d26 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:50.680163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:50.685646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0de897-70a6-45ae-b0b0-d71ebe610302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:50.686276 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0de897-70a6-45ae-b0b0-d71ebe610302 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:50.686760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0de897-70a6-45ae-b0b0-d71ebe610302 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:50.687035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0de897-70a6-45ae-b0b0-d71ebe610302 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:50.687835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0de897-70a6-45ae-b0b0-d71ebe610302 None admin] RBAC: Adding query filter params (group.id=dbae207e1080407fa50c901b073e99fa, role.id=bda5c75a326e4f5d9503960830898895, scope.project.id=393ec3a506734c8b8c02823cb21848e2) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:50.688148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0de897-70a6-45ae-b0b0-d71ebe610302 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:50.689687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0de897-70a6-45ae-b0b0-d71ebe610302 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:50.698182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 205/423] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Aug 11 09:40:50 2022] GET /identity/v3/role_assignments?group.id=dbae207e1080407fa50c901b073e99fa&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=bda5c75a326e4f5d9503960830898895 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:50.828266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9314c1-fde3-41df-9d61-81cde3bf79de None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:50.828824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9314c1-fde3-41df-9d61-81cde3bf79de None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:50.828824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9314c1-fde3-41df-9d61-81cde3bf79de None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:50.830584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 219/424] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:50.836838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd29a1-ab07-4b50-8492-6ef8f43c802e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:50.837170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd29a1-ab07-4b50-8492-6ef8f43c802e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:50.837420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd29a1-ab07-4b50-8492-6ef8f43c802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:50.865499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6abd29a1-ab07-4b50-8492-6ef8f43c802e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:50.866739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6abd29a1-ab07-4b50-8492-6ef8f43c802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:50.957893 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 206/425] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:50.981844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9314c1-fde3-41df-9d61-81cde3bf79de None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:50.992343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:51.094148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9314c1-fde3-41df-9d61-81cde3bf79de None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:51.134229 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca0432-7ca9-48fd-bb2b-f10dcf577ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:51.134862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca0432-7ca9-48fd-bb2b-f10dcf577ead None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:51.135199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca0432-7ca9-48fd-bb2b-f10dcf577ead None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:51.135477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca0432-7ca9-48fd-bb2b-f10dcf577ead None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:51.148511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca0432-7ca9-48fd-bb2b-f10dcf577ead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:51.152513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a1354-a052-45f2-b059-07dd907eaf3f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:51.152793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a1354-a052-45f2-b059-07dd907eaf3f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:51.153104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a1354-a052-45f2-b059-07dd907eaf3f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f8ca0432-7ca9-48fd-bb2b-f10dcf577ead None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:51.154475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:51.155446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 09:40:51.156308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:51.156927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 207/426] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:51.156993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 220/427] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:51.161461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca0432-7ca9-48fd-bb2b-f10dcf577ead None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:51.162022 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d737b8-dfd7-4451-b588-1a5384cc162b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:51.162801 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d737b8-dfd7-4451-b588-1a5384cc162b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:51.163363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d737b8-dfd7-4451-b588-1a5384cc162b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:51.200957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f2d737b8-dfd7-4451-b588-1a5384cc162b None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:51.202425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f2d737b8-dfd7-4451-b588-1a5384cc162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:51.258995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:51.300138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 208/428] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:51.303305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca0432-7ca9-48fd-bb2b-f10dcf577ead None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:51.331841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d737b8-dfd7-4451-b588-1a5384cc162b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:51.351837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea8a41-2a16-44d1-bb78-f7fd1d5271cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:51.352379 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea8a41-2a16-44d1-bb78-f7fd1d5271cc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:51.352659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea8a41-2a16-44d1-bb78-f7fd1d5271cc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:51.352928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea8a41-2a16-44d1-bb78-f7fd1d5271cc None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:51.353705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea8a41-2a16-44d1-bb78-f7fd1d5271cc None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:51.353993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea8a41-2a16-44d1-bb78-f7fd1d5271cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:51.355683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea8a41-2a16-44d1-bb78-f7fd1d5271cc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:51.366139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 221/429] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:40:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:51.374110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea8a41-2a16-44d1-bb78-f7fd1d5271cc None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:51.405150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:51.455850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d737b8-dfd7-4451-b588-1a5384cc162b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:51.497331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1132a8-54ab-44da-83fd-1f6c7cc3a800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:51.498305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1132a8-54ab-44da-83fd-1f6c7cc3a800 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:51.498665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1132a8-54ab-44da-83fd-1f6c7cc3a800 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:51.499006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1132a8-54ab-44da-83fd-1f6c7cc3a800 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:51.510243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1132a8-54ab-44da-83fd-1f6c7cc3a800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:51.511535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea8a41-2a16-44d1-bb78-f7fd1d5271cc None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-da1132a8-54ab-44da-83fd-1f6c7cc3a800 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:51.515300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:51.516350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:51.517338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:51.524096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 209/430] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:51.525597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1132a8-54ab-44da-83fd-1f6c7cc3a800 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:51.548800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:51.559646 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee9923-11c6-4045-afd0-0dbad53c7b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:51.560265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee9923-11c6-4045-afd0-0dbad53c7b82 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:51.560548 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee9923-11c6-4045-afd0-0dbad53c7b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:51.560881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee9923-11c6-4045-afd0-0dbad53c7b82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:51.646940 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1132a8-54ab-44da-83fd-1f6c7cc3a800 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:51.649879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee9923-11c6-4045-afd0-0dbad53c7b82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:51.651605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee9923-11c6-4045-afd0-0dbad53c7b82 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:51.665896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:51.684387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e75ef-f4d3-4b5e-bc78-90874dafb06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:51.685278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e75ef-f4d3-4b5e-bc78-90874dafb06e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:51.685387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e75ef-f4d3-4b5e-bc78-90874dafb06e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:51.685611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e75ef-f4d3-4b5e-bc78-90874dafb06e None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:51.686412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e75ef-f4d3-4b5e-bc78-90874dafb06e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:51.686637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e75ef-f4d3-4b5e-bc78-90874dafb06e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:51.688080 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e75ef-f4d3-4b5e-bc78-90874dafb06e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:51.697179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 210/431] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:51.702922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e75ef-f4d3-4b5e-bc78-90874dafb06e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:51.779015 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 222/432] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:51.786029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee9923-11c6-4045-afd0-0dbad53c7b82 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:51.814473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e75ef-f4d3-4b5e-bc78-90874dafb06e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:51.815059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:51.856238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f994f0d-8387-45ee-b5c4-9378b564b127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:51.856852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f994f0d-8387-45ee-b5c4-9378b564b127 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:51.857174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f994f0d-8387-45ee-b5c4-9378b564b127 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:51.857447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f994f0d-8387-45ee-b5c4-9378b564b127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:51.891226 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee9923-11c6-4045-afd0-0dbad53c7b82 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:51.908754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:51.929719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fab82-6d9a-4416-878d-4bbee4ee4f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:51.930214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f994f0d-8387-45ee-b5c4-9378b564b127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:51.930384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fab82-6d9a-4416-878d-4bbee4ee4f4e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:51.930630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fab82-6d9a-4416-878d-4bbee4ee4f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:51.930949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fab82-6d9a-4416-878d-4bbee4ee4f4e None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:51.931771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f994f0d-8387-45ee-b5c4-9378b564b127 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:51.932103 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fab82-6d9a-4416-878d-4bbee4ee4f4e None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:51.933580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fab82-6d9a-4416-878d-4bbee4ee4f4e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:51.939649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 223/433] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:51 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:51.939649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51ebdd-2ad8-4ede-ad10-399babbebeb4 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:51.939649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51ebdd-2ad8-4ede-ad10-399babbebeb4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:51.939649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51ebdd-2ad8-4ede-ad10-399babbebeb4 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:51.939649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 224/434] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:51.940419 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51ebdd-2ad8-4ede-ad10-399babbebeb4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:52.041885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:52.049371 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51ebdd-2ad8-4ede-ad10-399babbebeb4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:52.050185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 211/435] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 09:40:52.051299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf78fa0-952f-4452-9cc0-b12e71eaabf6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.051734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf78fa0-952f-4452-9cc0-b12e71eaabf6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.052143 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf78fa0-952f-4452-9cc0-b12e71eaabf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.072888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-dcf78fa0-952f-4452-9cc0-b12e71eaabf6 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:52.074155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dcf78fa0-952f-4452-9cc0-b12e71eaabf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:52.089807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbc74f-3366-428a-87ab-220a08441291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:52.090440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbc74f-3366-428a-87ab-220a08441291 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.090711 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbc74f-3366-428a-87ab-220a08441291 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.090982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbc74f-3366-428a-87ab-220a08441291 None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.092088 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbc74f-3366-428a-87ab-220a08441291 None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:52.093524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbc74f-3366-428a-87ab-220a08441291 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:52.100189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 225/436] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:51 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:52.100189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbc74f-3366-428a-87ab-220a08441291 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:52.161112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 212/437] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:52.161112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf78fa0-952f-4452-9cc0-b12e71eaabf6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:52.171104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:52.224795 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbc74f-3366-428a-87ab-220a08441291 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:52.263334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaf3cb-3c9e-49af-a621-4d0a30afdacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:52.263964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf3cb-3c9e-49af-a621-4d0a30afdacd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.264240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf3cb-3c9e-49af-a621-4d0a30afdacd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.264513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf3cb-3c9e-49af-a621-4d0a30afdacd None admin] PATH_INFO: `/v3/projects/393ec3a506734c8b8c02823cb21848e2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.265673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaf3cb-3c9e-49af-a621-4d0a30afdacd None admin] RBAC: Authorizing `identity:get_project(project_id=393ec3a506734c8b8c02823cb21848e2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:52.267117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaf3cb-3c9e-49af-a621-4d0a30afdacd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:52.273472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 226/438] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:52 2022] GET /identity/v3/projects/393ec3a506734c8b8c02823cb21848e2 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:52.273472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaf3cb-3c9e-49af-a621-4d0a30afdacd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:52.289990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf78fa0-952f-4452-9cc0-b12e71eaabf6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:52.304844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:52.335789 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928604ae-7e28-469e-b987-8d485760d3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:52.336461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928604ae-7e28-469e-b987-8d485760d3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.336881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928604ae-7e28-469e-b987-8d485760d3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.337210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928604ae-7e28-469e-b987-8d485760d3c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.338056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928604ae-7e28-469e-b987-8d485760d3c1 None admin] RBAC: Adding query filter params (group.id=dbae207e1080407fa50c901b073e99fa, role.id=bda5c75a326e4f5d9503960830898895, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:52.338424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928604ae-7e28-469e-b987-8d485760d3c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:52.340172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928604ae-7e28-469e-b987-8d485760d3c1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:52.357064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 213/439] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Aug 11 09:40:52 2022] GET /identity/v3/role_assignments?group.id=dbae207e1080407fa50c901b073e99fa&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=bda5c75a326e4f5d9503960830898895 => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:52.357064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928604ae-7e28-469e-b987-8d485760d3c1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:52.387309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaf3cb-3c9e-49af-a621-4d0a30afdacd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:52.430840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da33049-03f8-4ea9-abd4-4a4f67e6d630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:52.431578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da33049-03f8-4ea9-abd4-4a4f67e6d630 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.431856 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da33049-03f8-4ea9-abd4-4a4f67e6d630 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.432752 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da33049-03f8-4ea9-abd4-4a4f67e6d630 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.436094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:52.448094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da33049-03f8-4ea9-abd4-4a4f67e6d630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4da33049-03f8-4ea9-abd4-4a4f67e6d630 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:52.453034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:52.454009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:52.454813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 227/440] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:52.456604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84355a8-300f-4011-bed2-234f3aff5d01 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.456604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84355a8-300f-4011-bed2-234f3aff5d01 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.456604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84355a8-300f-4011-bed2-234f3aff5d01 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.457490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 228/441] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:52.462799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84355a8-300f-4011-bed2-234f3aff5d01 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:52.477224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928604ae-7e28-469e-b987-8d485760d3c1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:52.520584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568f877-f4b2-46d9-9069-30037b94ab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:52.520584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568f877-f4b2-46d9-9069-30037b94ab2f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.520878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568f877-f4b2-46d9-9069-30037b94ab2f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.520965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568f877-f4b2-46d9-9069-30037b94ab2f None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.522664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568f877-f4b2-46d9-9069-30037b94ab2f None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:52.524102 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568f877-f4b2-46d9-9069-30037b94ab2f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:52.525690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 214/442] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:52 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:52.526795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb79d3-b96e-4ba0-9f0d-29f7d077a6c5 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.527079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb79d3-b96e-4ba0-9f0d-29f7d077a6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.527347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb79d3-b96e-4ba0-9f0d-29f7d077a6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.548075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e9eb79d3-b96e-4ba0-9f0d-29f7d077a6c5 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:52.549352 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e9eb79d3-b96e-4ba0-9f0d-29f7d077a6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:52.579124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:52.588290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84355a8-300f-4011-bed2-234f3aff5d01 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:52.623863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed851-df7f-490e-b9db-c409623ffc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:52.624474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed851-df7f-490e-b9db-c409623ffc7f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.624755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed851-df7f-490e-b9db-c409623ffc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.625047 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed851-df7f-490e-b9db-c409623ffc7f None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.625714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eed851-df7f-490e-b9db-c409623ffc7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:52.626022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eed851-df7f-490e-b9db-c409623ffc7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:52.627385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eed851-df7f-490e-b9db-c409623ffc7f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:52.631377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 215/443] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:52.632220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb79d3-b96e-4ba0-9f0d-29f7d077a6c5 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:52.652795 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 229/444] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:52.653948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed851-df7f-490e-b9db-c409623ffc7f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:52.703242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:52.734034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb79d3-b96e-4ba0-9f0d-29f7d077a6c5 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:52.752130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed851-df7f-490e-b9db-c409623ffc7f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:52.767549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d35c1-fd38-4ef7-aa86-8f3f28e36c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:52.768151 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d35c1-fd38-4ef7-aa86-8f3f28e36c76 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.768435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d35c1-fd38-4ef7-aa86-8f3f28e36c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.768710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d35c1-fd38-4ef7-aa86-8f3f28e36c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.769469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d35c1-fd38-4ef7-aa86-8f3f28e36c76 None admin] RBAC: Adding query filter params (group.id=dbae207e1080407fa50c901b073e99fa, role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596, scope.project.id=393ec3a506734c8b8c02823cb21848e2) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:52.769775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d35c1-fd38-4ef7-aa86-8f3f28e36c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:52.772093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d35c1-fd38-4ef7-aa86-8f3f28e36c76 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:52.777695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 216/445] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Aug 11 09:40:52 2022] GET /identity/v3/role_assignments?group.id=dbae207e1080407fa50c901b073e99fa&scope.project.id=393ec3a506734c8b8c02823cb21848e2&role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:52.778601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d35c1-fd38-4ef7-aa86-8f3f28e36c76 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:52.788870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebf862-d8b4-40c8-b461-d461c7c4b124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:52.789581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebf862-d8b4-40c8-b461-d461c7c4b124 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.789868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebf862-d8b4-40c8-b461-d461c7c4b124 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.790143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebf862-d8b4-40c8-b461-d461c7c4b124 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.802081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebf862-d8b4-40c8-b461-d461c7c4b124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-37ebf862-d8b4-40c8-b461-d461c7c4b124 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:52.808543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:52.809537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:52.810323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 230/446] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:40:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:52.815933 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebf862-d8b4-40c8-b461-d461c7c4b124 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:52.843955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:52.910966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d35c1-fd38-4ef7-aa86-8f3f28e36c76 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:52.946191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebf862-d8b4-40c8-b461-d461c7c4b124 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:52.950092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6031f-5bbd-4f41-a91e-44603c897c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:52.950702 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6031f-5bbd-4f41-a91e-44603c897c32 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.950979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6031f-5bbd-4f41-a91e-44603c897c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.951250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6031f-5bbd-4f41-a91e-44603c897c32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.954124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:52.994078 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64665af-6e72-4374-8833-46c7c935f09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:52.994485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64665af-6e72-4374-8833-46c7c935f09b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:52.994771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64665af-6e72-4374-8833-46c7c935f09b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:52.995045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64665af-6e72-4374-8833-46c7c935f09b None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:52.995721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64665af-6e72-4374-8833-46c7c935f09b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:52.996029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64665af-6e72-4374-8833-46c7c935f09b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:52.997445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64665af-6e72-4374-8833-46c7c935f09b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:53.007158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 231/447] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:40:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:53.012604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64665af-6e72-4374-8833-46c7c935f09b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:53.035000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6031f-5bbd-4f41-a91e-44603c897c32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:53.036568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6031f-5bbd-4f41-a91e-44603c897c32 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:53.107961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:53.135961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64665af-6e72-4374-8833-46c7c935f09b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:53.148622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 217/448] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:40:53.154649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6031f-5bbd-4f41-a91e-44603c897c32 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:53.185855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5359c-da3a-45d7-a7af-79e9a1d0121f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:53.186370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5359c-da3a-45d7-a7af-79e9a1d0121f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:53.186682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5359c-da3a-45d7-a7af-79e9a1d0121f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:53.186986 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5359c-da3a-45d7-a7af-79e9a1d0121f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:53.200399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:53.266287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5359c-da3a-45d7-a7af-79e9a1d0121f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:53.267636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5359c-da3a-45d7-a7af-79e9a1d0121f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:53.288689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6031f-5bbd-4f41-a91e-44603c897c32 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:53.324554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474e57c-64d1-4372-abcb-c19f508f26da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:53.325225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e57c-64d1-4372-abcb-c19f508f26da None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:53.325503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e57c-64d1-4372-abcb-c19f508f26da None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:53.325770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e57c-64d1-4372-abcb-c19f508f26da None admin] PATH_INFO: `/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:53.326882 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474e57c-64d1-4372-abcb-c19f508f26da None admin] RBAC: Authorizing `identity:get_project(project_id=2fb27f0238c0484ca533d76cafdcc6d6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:53.328397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474e57c-64d1-4372-abcb-c19f508f26da None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:53.330062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 218/449] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:53 2022] GET /identity/v3/projects/2fb27f0238c0484ca533d76cafdcc6d6 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:53.335396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474e57c-64d1-4372-abcb-c19f508f26da None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:53.352483 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:53.376008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 232/450] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:40:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 09:40:53.381579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5359c-da3a-45d7-a7af-79e9a1d0121f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:53.434227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474e57c-64d1-4372-abcb-c19f508f26da None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:53.435738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:53.468712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830185c-d40c-4113-a9fa-58bee7abd855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:53.469371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830185c-d40c-4113-a9fa-58bee7abd855 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:53.469650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830185c-d40c-4113-a9fa-58bee7abd855 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:53.469922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830185c-d40c-4113-a9fa-58bee7abd855 None admin] PATH_INFO: `/v3/groups/dbae207e1080407fa50c901b073e99fa` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:53.471053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830185c-d40c-4113-a9fa-58bee7abd855 None admin] RBAC: Authorizing `identity:get_group(group_id=dbae207e1080407fa50c901b073e99fa)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:53.472499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830185c-d40c-4113-a9fa-58bee7abd855 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:53.474879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 219/451] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:53 2022] GET /identity/v3/groups/dbae207e1080407fa50c901b073e99fa => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:53.480185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5359c-da3a-45d7-a7af-79e9a1d0121f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:53.480721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830185c-d40c-4113-a9fa-58bee7abd855 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:53.513792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80d661-15bc-4de4-8725-d2f32e1545f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:53.514420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80d661-15bc-4de4-8725-d2f32e1545f8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:53.514704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80d661-15bc-4de4-8725-d2f32e1545f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:53.514981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80d661-15bc-4de4-8725-d2f32e1545f8 None admin] PATH_INFO: `/v3/projects/c055f43eb0f74f81a1a2e702ea1b7964` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:53.516108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80d661-15bc-4de4-8725-d2f32e1545f8 None admin] RBAC: Authorizing `identity:get_project(project_id=c055f43eb0f74f81a1a2e702ea1b7964)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:53.517623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80d661-15bc-4de4-8725-d2f32e1545f8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:53.519190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 233/452] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:40:53 2022] GET /identity/v3/projects/c055f43eb0f74f81a1a2e702ea1b7964 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:53.525139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80d661-15bc-4de4-8725-d2f32e1545f8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:53.569787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:53.580336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830185c-d40c-4113-a9fa-58bee7abd855 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:53.615197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617a2e2-f134-4362-92a4-a516ad7f8dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:53.615796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a2e2-f134-4362-92a4-a516ad7f8dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:53.616071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a2e2-f134-4362-92a4-a516ad7f8dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:53.616340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617a2e2-f134-4362-92a4-a516ad7f8dd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:53.617094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617a2e2-f134-4362-92a4-a516ad7f8dd3 None admin] RBAC: Adding query filter params (group.id=dbae207e1080407fa50c901b073e99fa, role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596, scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:53.617399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617a2e2-f134-4362-92a4-a516ad7f8dd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:53.618815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617a2e2-f134-4362-92a4-a516ad7f8dd3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:53.623853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80d661-15bc-4de4-8725-d2f32e1545f8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:53.624429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 220/453] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Aug 11 09:40:53 2022] GET /identity/v3/role_assignments?group.id=dbae207e1080407fa50c901b073e99fa&scope.project.id=2fb27f0238c0484ca533d76cafdcc6d6&role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:53.659529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258f577-b5f0-4d4b-9bd1-7528f32f2a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:53.660179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258f577-b5f0-4d4b-9bd1-7528f32f2a9f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:53.660457 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258f577-b5f0-4d4b-9bd1-7528f32f2a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:53.660725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258f577-b5f0-4d4b-9bd1-7528f32f2a9f None admin] PATH_INFO: `/v3/groups/1378a8a95dd949649fc2c9d49b45133e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:53.662066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258f577-b5f0-4d4b-9bd1-7528f32f2a9f None admin] RBAC: Authorizing `identity:get_group(group_id=1378a8a95dd949649fc2c9d49b45133e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:53.663508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258f577-b5f0-4d4b-9bd1-7528f32f2a9f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:53.665210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 234/454] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Aug 11 09:40:53 2022] GET /identity/v3/groups/1378a8a95dd949649fc2c9d49b45133e => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:53.671281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617a2e2-f134-4362-92a4-a516ad7f8dd3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:53.708464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:53.777677 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617a2e2-f134-4362-92a4-a516ad7f8dd3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:53.813186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fdc01-f2e6-4ec7-9d51-8bf05fcdeb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:53.813790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fdc01-f2e6-4ec7-9d51-8bf05fcdeb56 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:53.814066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fdc01-f2e6-4ec7-9d51-8bf05fcdeb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:53.814335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fdc01-f2e6-4ec7-9d51-8bf05fcdeb56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:53.815061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fdc01-f2e6-4ec7-9d51-8bf05fcdeb56 None admin] RBAC: Adding query filter params (group.id=1378a8a95dd949649fc2c9d49b45133e, role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=c055f43eb0f74f81a1a2e702ea1b7964) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:53.815360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fdc01-f2e6-4ec7-9d51-8bf05fcdeb56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:53.816790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fdc01-f2e6-4ec7-9d51-8bf05fcdeb56 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:53.822967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 221/455] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Aug 11 09:40:53 2022] GET /identity/v3/role_assignments?group.id=1378a8a95dd949649fc2c9d49b45133e&scope.project.id=c055f43eb0f74f81a1a2e702ea1b7964&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:56.248569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04cd7-3eb9-461e-88bf-2b07eb32f6d4 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:56.249479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04cd7-3eb9-461e-88bf-2b07eb32f6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:56.249951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c04cd7-3eb9-461e-88bf-2b07eb32f6d4 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:56.251797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 235/456] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:56.256723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bfb05-4e6b-4bf3-ab27-10b15bb9509e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:56.257571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bfb05-4e6b-4bf3-ab27-10b15bb9509e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:56.258070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bfb05-4e6b-4bf3-ab27-10b15bb9509e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:56.286999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c46bfb05-4e6b-4bf3-ab27-10b15bb9509e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:56.288867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c46bfb05-4e6b-4bf3-ab27-10b15bb9509e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:56.375038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 222/457] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:56.412617 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c04cd7-3eb9-461e-88bf-2b07eb32f6d4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:56.480261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06bb92-a6e0-40d5-9f1c-aaca247afb79 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:56.481380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06bb92-a6e0-40d5-9f1c-aaca247afb79 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:56.481874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06bb92-a6e0-40d5-9f1c-aaca247afb79 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:56.484042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 223/458] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:56.490148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67ca03-f25f-4212-baca-1e012e472102 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:56.490805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67ca03-f25f-4212-baca-1e012e472102 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:56.491666 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67ca03-f25f-4212-baca-1e012e472102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:56.513293 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-dc67ca03-f25f-4212-baca-1e012e472102 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:56.514832 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dc67ca03-f25f-4212-baca-1e012e472102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:56.557434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:56.593686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 224/459] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:56.620106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67ca03-f25f-4212-baca-1e012e472102 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:56.648018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c04cd7-3eb9-461e-88bf-2b07eb32f6d4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:56.675599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:56.698881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03467b5e-fa0d-4781-be35-1d725934a945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:56.699528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03467b5e-fa0d-4781-be35-1d725934a945 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:56.700037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03467b5e-fa0d-4781-be35-1d725934a945 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:56.700480 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03467b5e-fa0d-4781-be35-1d725934a945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:56.711990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03467b5e-fa0d-4781-be35-1d725934a945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:56.713709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03467b5e-fa0d-4781-be35-1d725934a945 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-03467b5e-fa0d-4781-be35-1d725934a945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:40:56.730392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:56.731280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:40:56.732111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:40:56.732111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:40:56.732111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:56.734140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 236/460] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:56.735230 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67ca03-f25f-4212-baca-1e012e472102 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:56.735711 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b1612-5e86-49ad-9297-96ce94c4d696 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:56.736191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b1612-5e86-49ad-9297-96ce94c4d696 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:56.736651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b1612-5e86-49ad-9297-96ce94c4d696 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:56.738157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 237/461] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:56.739653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b1612-5e86-49ad-9297-96ce94c4d696 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:56.781606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af618490-6396-4488-aba2-7c83596f6ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:56.782233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af618490-6396-4488-aba2-7c83596f6ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:56.782515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af618490-6396-4488-aba2-7c83596f6ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:56.782784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af618490-6396-4488-aba2-7c83596f6ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:56.796926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af618490-6396-4488-aba2-7c83596f6ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:56.799170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af618490-6396-4488-aba2-7c83596f6ca8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-af618490-6396-4488-aba2-7c83596f6ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:40:56.819922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:56.820851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:40:56.821735 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:40:56.821735 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:40:56.821735 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:56.825164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 225/462] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:56.826619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d417df-398b-4f6e-b450-7e2ce706b235 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:56.826899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d417df-398b-4f6e-b450-7e2ce706b235 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:56.827170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d417df-398b-4f6e-b450-7e2ce706b235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:56.846568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-19d417df-398b-4f6e-b450-7e2ce706b235 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:56.847754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-19d417df-398b-4f6e-b450-7e2ce706b235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:56.851982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:56.863517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b1612-5e86-49ad-9297-96ce94c4d696 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:56.908539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42d47a-8299-4427-8870-8edeaf067660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:56.909732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42d47a-8299-4427-8870-8edeaf067660 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:56.910014 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42d47a-8299-4427-8870-8edeaf067660 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:56.910284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42d47a-8299-4427-8870-8edeaf067660 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:56.910930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42d47a-8299-4427-8870-8edeaf067660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:56.911962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42d47a-8299-4427-8870-8edeaf067660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:56.913375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42d47a-8299-4427-8870-8edeaf067660 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:56.925279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 226/463] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:56.925946 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 238/464] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:56.926751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d417df-398b-4f6e-b450-7e2ce706b235 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:56.938126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42d47a-8299-4427-8870-8edeaf067660 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:56.970598 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:57.040364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d417df-398b-4f6e-b450-7e2ce706b235 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:57.052826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42d47a-8299-4427-8870-8edeaf067660 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:57.077351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace774b-3ffe-47e0-b4ee-6c5aabcd73c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:57.078070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace774b-3ffe-47e0-b4ee-6c5aabcd73c3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:57.078345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace774b-3ffe-47e0-b4ee-6c5aabcd73c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:57.078726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace774b-3ffe-47e0-b4ee-6c5aabcd73c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:57.079359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace774b-3ffe-47e0-b4ee-6c5aabcd73c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:57.079658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace774b-3ffe-47e0-b4ee-6c5aabcd73c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:57.081212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace774b-3ffe-47e0-b4ee-6c5aabcd73c3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:57.086387 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da984c-cba4-4535-ab4a-e16ac16b1a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:57.086989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da984c-cba4-4535-ab4a-e16ac16b1a9e None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:57.087266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da984c-cba4-4535-ab4a-e16ac16b1a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:57.087601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da984c-cba4-4535-ab4a-e16ac16b1a9e None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:57.088203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da984c-cba4-4535-ab4a-e16ac16b1a9e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:57.089643 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da984c-cba4-4535-ab4a-e16ac16b1a9e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:57.095664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:57.100692 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 227/465] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:57.105960 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace774b-3ffe-47e0-b4ee-6c5aabcd73c3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:57.200421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 239/466] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:40:56 2022] POST /identity/v3/users => generated 246 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:40:57.201723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397727d-3776-45e2-89dd-e58730b43bc7 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:57.202108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397727d-3776-45e2-89dd-e58730b43bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:57.202482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397727d-3776-45e2-89dd-e58730b43bc7 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:57.204090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 240/467] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:57.205199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397727d-3776-45e2-89dd-e58730b43bc7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:57.211838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace774b-3ffe-47e0-b4ee-6c5aabcd73c3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:57.252071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd329c-2127-4a45-989a-1761014aa44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:57.253115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd329c-2127-4a45-989a-1761014aa44e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:57.253592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd329c-2127-4a45-989a-1761014aa44e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:57.254087 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd329c-2127-4a45-989a-1761014aa44e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:57.272083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd329c-2127-4a45-989a-1761014aa44e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:57.274321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd329c-2127-4a45-989a-1761014aa44e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-befd329c-2127-4a45-989a-1761014aa44e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:40:57.291360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:57.292291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:40:57.293127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:40:57.293127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:40:57.293127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:57.295270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 228/468] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:57.296858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cfb42-bb35-469f-9dec-2408e50c31c8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:57.297193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cfb42-bb35-469f-9dec-2408e50c31c8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:57.297445 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cfb42-bb35-469f-9dec-2408e50c31c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:57.316465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7d2cfb42-bb35-469f-9dec-2408e50c31c8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:57.318054 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7d2cfb42-bb35-469f-9dec-2408e50c31c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:57.323607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:57.350377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397727d-3776-45e2-89dd-e58730b43bc7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:57.384658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:57.404544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4beb6b-140a-4510-98a4-316367e6f338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:57.405226 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4beb6b-140a-4510-98a4-316367e6f338 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:57.405506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4beb6b-140a-4510-98a4-316367e6f338 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:57.405802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4beb6b-140a-4510-98a4-316367e6f338 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:57.407255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4beb6b-140a-4510-98a4-316367e6f338 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:57.413975 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 229/469] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:57.414213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4beb6b-140a-4510-98a4-316367e6f338 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:57.414293 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cfb42-bb35-469f-9dec-2408e50c31c8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:57.515522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 241/470] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:40:57 2022] POST /identity/v3/users => generated 248 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:40:57.516507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4beb6b-140a-4510-98a4-316367e6f338 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:57.528382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cfb42-bb35-469f-9dec-2408e50c31c8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:57.605225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e7df7-1e7b-46e1-83f5-b071d070cee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:57.605770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e7df7-1e7b-46e1-83f5-b071d070cee3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:57.606050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e7df7-1e7b-46e1-83f5-b071d070cee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:57.606380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e7df7-1e7b-46e1-83f5-b071d070cee3 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:57.607053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e7df7-1e7b-46e1-83f5-b071d070cee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:57.607357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e7df7-1e7b-46e1-83f5-b071d070cee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:57.608801 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e7df7-1e7b-46e1-83f5-b071d070cee3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:57.632115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 230/471] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:57.640136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e7df7-1e7b-46e1-83f5-b071d070cee3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:57.707865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4beb6b-140a-4510-98a4-316367e6f338 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:57.720010 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:57.756744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d8056-9fee-49b1-94fc-acf20cc1cd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:57.757425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d8056-9fee-49b1-94fc-acf20cc1cd2d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:57.757709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d8056-9fee-49b1-94fc-acf20cc1cd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:57.757983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d8056-9fee-49b1-94fc-acf20cc1cd2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:57.761446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:57.774577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d8056-9fee-49b1-94fc-acf20cc1cd2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:57.776297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d8056-9fee-49b1-94fc-acf20cc1cd2d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:57.787598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e7df7-1e7b-46e1-83f5-b071d070cee3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f15d8056-9fee-49b1-94fc-acf20cc1cd2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:40:57.798022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:57.801003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:40:57.801899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:40:57.801899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:40:57.801899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:57.801899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 242/472] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:57.805870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d8056-9fee-49b1-94fc-acf20cc1cd2d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:57.836862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ec176-e31b-47db-8c87-dad4ec03c490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:57.837854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ec176-e31b-47db-8c87-dad4ec03c490 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:57.838141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ec176-e31b-47db-8c87-dad4ec03c490 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:57.838470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ec176-e31b-47db-8c87-dad4ec03c490 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:57.839277 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ec176-e31b-47db-8c87-dad4ec03c490 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:57.841418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ec176-e31b-47db-8c87-dad4ec03c490 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:57.925243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d8056-9fee-49b1-94fc-acf20cc1cd2d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:57.944847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 231/473] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:40:57 2022] POST /identity/v3/users => generated 248 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:40:57.971452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea551d-0a55-460e-bd4e-32fdab7817ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:57.972073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea551d-0a55-460e-bd4e-32fdab7817ad None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:57.972358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea551d-0a55-460e-bd4e-32fdab7817ad None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:57.972636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea551d-0a55-460e-bd4e-32fdab7817ad None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:57.973323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea551d-0a55-460e-bd4e-32fdab7817ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:57.973630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea551d-0a55-460e-bd4e-32fdab7817ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:57.975895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea551d-0a55-460e-bd4e-32fdab7817ad None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:57.995090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 243/474] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:40:57.996488 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ec176-e31b-47db-8c87-dad4ec03c490 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:58.006416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaeb45-ce6b-4986-8332-b5503fdc2bd9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:58.006757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaeb45-ce6b-4986-8332-b5503fdc2bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:58.006972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaeb45-ce6b-4986-8332-b5503fdc2bd9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:58.008692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 244/475] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:40:58.014908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732249a-271c-438f-9d3a-ef9b6aad79da None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:58.015198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732249a-271c-438f-9d3a-ef9b6aad79da None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:58.015472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732249a-271c-438f-9d3a-ef9b6aad79da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:58.046570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5732249a-271c-438f-9d3a-ef9b6aad79da None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:58.047863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5732249a-271c-438f-9d3a-ef9b6aad79da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:58.138555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:58.149720 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:58.157474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ec176-e31b-47db-8c87-dad4ec03c490 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:58.172037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 245/476] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:40:58.206587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732249a-271c-438f-9d3a-ef9b6aad79da None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:58.224705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6fcc8-7816-48db-8a75-5d9c3e9a8340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:58.225732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fcc8-7816-48db-8a75-5d9c3e9a8340 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:58.226176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fcc8-7816-48db-8a75-5d9c3e9a8340 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:58.226660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fcc8-7816-48db-8a75-5d9c3e9a8340 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:58.227638 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6fcc8-7816-48db-8a75-5d9c3e9a8340 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:58.229385 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6fcc8-7816-48db-8a75-5d9c3e9a8340 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:58.315132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 232/477] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:40:57 2022] POST /identity/v3/users => generated 249 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:40:58.329949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732249a-271c-438f-9d3a-ef9b6aad79da None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:58.373203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0720251-029e-4030-87c7-9c3dc50f9712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:58.373946 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0720251-029e-4030-87c7-9c3dc50f9712 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:58.374176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0720251-029e-4030-87c7-9c3dc50f9712 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:58.374464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0720251-029e-4030-87c7-9c3dc50f9712 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:58.375151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0720251-029e-4030-87c7-9c3dc50f9712 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:58.376624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0720251-029e-4030-87c7-9c3dc50f9712 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:58.423960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.sql.core [None req-f0720251-029e-4030-87c7-9c3dc50f9712 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 11 09:40:58.423960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 11 09:40:58.423960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [parameters: {'id': '6002ec181aec447a9ba20fa14567a752', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 11 09:40:58.423960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81638) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f0720251-029e-4030-87c7-9c3dc50f9712 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application conn.query(q) Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result.read() Aug 11 09:40:58.448840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return ref.to_dict() Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 09:40:58.449807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application next(self.gen) Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application yield resource Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application next(self.gen) Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.session.rollback() Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.force_reraise() Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise self.value Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application session.commit() Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 11 09:40:58.450713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._prepare_impl() Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.session.flush() Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._flush(objects) Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application compat.raise_( Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application flush_context.execute() Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rec.execute(self) Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 11 09:40:58.451540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 09:40:58.452404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application conn.query(q) Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result.read() Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 09:40:58.454426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [parameters: {'id': '6002ec181aec447a9ba20fa14567a752', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:58.456496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 11 09:40:58.465222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:40:58.467502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 246/478] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Aug 11 09:40:58 2022] POST /identity/v3/roles => generated 144 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:40:58.467648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6fcc8-7816-48db-8a75-5d9c3e9a8340 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:58.590684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6fcc8-7816-48db-8a75-5d9c3e9a8340 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:58.632263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b9936-dbe0-445c-a7a9-fae611177191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:58.633323 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b9936-dbe0-445c-a7a9-fae611177191 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:58.633805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b9936-dbe0-445c-a7a9-fae611177191 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:58.634298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b9936-dbe0-445c-a7a9-fae611177191 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:58.645351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b9936-dbe0-445c-a7a9-fae611177191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-593b9936-dbe0-445c-a7a9-fae611177191 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:58.650941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:58.651816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 09:40:58.652587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:58.655095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 233/479] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:40:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:58.660492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0720251-029e-4030-87c7-9c3dc50f9712 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:58.711705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:58.776340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0720251-029e-4030-87c7-9c3dc50f9712 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:58.813751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec55928-bdd2-4655-9fd3-b48b3b51df5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:58.814375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec55928-bdd2-4655-9fd3-b48b3b51df5a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:58.814655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec55928-bdd2-4655-9fd3-b48b3b51df5a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:58.814927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec55928-bdd2-4655-9fd3-b48b3b51df5a None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:58.815601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec55928-bdd2-4655-9fd3-b48b3b51df5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:40:58.815903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec55928-bdd2-4655-9fd3-b48b3b51df5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:40:58.817663 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec55928-bdd2-4655-9fd3-b48b3b51df5a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:40:58.826469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 247/480] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:40:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:40:59.509490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc2735-6295-42eb-b2d1-f656367f3c43 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:59.510135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc2735-6295-42eb-b2d1-f656367f3c43 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:59.510135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc2735-6295-42eb-b2d1-f656367f3c43 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:59.511525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 234/481] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:40:59.516932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64fbdc-0243-4ce0-8fab-f98a206ad3a7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:59.517360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64fbdc-0243-4ce0-8fab-f98a206ad3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:59.517544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64fbdc-0243-4ce0-8fab-f98a206ad3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:59.538587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-eb64fbdc-0243-4ce0-8fab-f98a206ad3a7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:40:59.539774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-eb64fbdc-0243-4ce0-8fab-f98a206ad3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:40:59.625544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 248/482] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:40:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:40:59.658560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc2735-6295-42eb-b2d1-f656367f3c43 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:59.677146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:40:59.793915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc2735-6295-42eb-b2d1-f656367f3c43 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:40:59.843451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff1414-5240-40de-a994-4a4bc8b51117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:40:59.844092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff1414-5240-40de-a994-4a4bc8b51117 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:40:59.844373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff1414-5240-40de-a994-4a4bc8b51117 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:40:59.844648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff1414-5240-40de-a994-4a4bc8b51117 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:40:59.858750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff1414-5240-40de-a994-4a4bc8b51117 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-8cff1414-5240-40de-a994-4a4bc8b51117 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:40:59.863611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:40:59.864646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:40:59.865740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 235/483] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:40:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:40:59.870603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64fbdc-0243-4ce0-8fab-f98a206ad3a7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:40:59.932250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:00.023045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64fbdc-0243-4ce0-8fab-f98a206ad3a7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:00.075199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343f106-0169-4dfa-952a-e285db5a99bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:00.076168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343f106-0169-4dfa-952a-e285db5a99bc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:00.076649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343f106-0169-4dfa-952a-e285db5a99bc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:00.077175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343f106-0169-4dfa-952a-e285db5a99bc None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:00.078072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343f106-0169-4dfa-952a-e285db5a99bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:00.078584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343f106-0169-4dfa-952a-e285db5a99bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:00.080149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343f106-0169-4dfa-952a-e285db5a99bc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:00.087210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2943fd-fe2f-4286-b604-ebd2a020da72 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:00.087498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2943fd-fe2f-4286-b604-ebd2a020da72 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:00.087774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2943fd-fe2f-4286-b604-ebd2a020da72 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:00.089178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 236/484] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:00.093843 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 249/485] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:40:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:00.094509 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f05f5-cd0a-4023-a9eb-f6748161a161 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:00.094689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f05f5-cd0a-4023-a9eb-f6748161a161 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:00.094953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f05f5-cd0a-4023-a9eb-f6748161a161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:00.099337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343f106-0169-4dfa-952a-e285db5a99bc None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:00.113948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-159f05f5-cd0a-4023-a9eb-f6748161a161 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:00.115245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-159f05f5-cd0a-4023-a9eb-f6748161a161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:00.213774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 237/486] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:00.245065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:00.251123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343f106-0169-4dfa-952a-e285db5a99bc None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:00.255128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f05f5-cd0a-4023-a9eb-f6748161a161 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:00.260152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:00.331651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b0abe-46c3-49cb-8cbd-a5974643bfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:00.332209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b0abe-46c3-49cb-8cbd-a5974643bfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:00.332491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b0abe-46c3-49cb-8cbd-a5974643bfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:00.332769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b0abe-46c3-49cb-8cbd-a5974643bfa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:00.348816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b0abe-46c3-49cb-8cbd-a5974643bfa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:00.351820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b0abe-46c3-49cb-8cbd-a5974643bfa7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6f1b0abe-46c3-49cb-8cbd-a5974643bfa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:00.377195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:00.378119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:00.378930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:00.378930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:00.378930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:00.378930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 250/487] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:00.385727 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b0abe-46c3-49cb-8cbd-a5974643bfa7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:00.431564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f05f5-cd0a-4023-a9eb-f6748161a161 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:00.469202 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644a72e-82ba-49b5-8d63-85ed0c5396e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:00.470204 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644a72e-82ba-49b5-8d63-85ed0c5396e8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:00.470738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644a72e-82ba-49b5-8d63-85ed0c5396e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:00.471211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0644a72e-82ba-49b5-8d63-85ed0c5396e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:00.482502 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0644a72e-82ba-49b5-8d63-85ed0c5396e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-0644a72e-82ba-49b5-8d63-85ed0c5396e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:00.487522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:00.488456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:00.489275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:00.491954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 238/488] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:00.502268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644a72e-82ba-49b5-8d63-85ed0c5396e8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:00.548740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b0abe-46c3-49cb-8cbd-a5974643bfa7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:00.587516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:00.619669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152b05c-e335-4e73-a40e-8064fadfbfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:00.620381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152b05c-e335-4e73-a40e-8064fadfbfe7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:00.620561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152b05c-e335-4e73-a40e-8064fadfbfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:00.620832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152b05c-e335-4e73-a40e-8064fadfbfe7 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:00.629035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152b05c-e335-4e73-a40e-8064fadfbfe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:00.629035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152b05c-e335-4e73-a40e-8064fadfbfe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:00.629035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0152b05c-e335-4e73-a40e-8064fadfbfe7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:00.636811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 251/489] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:00.638919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0644a72e-82ba-49b5-8d63-85ed0c5396e8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:00.646111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152b05c-e335-4e73-a40e-8064fadfbfe7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:00.667462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:00.676926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d6ace-19be-4580-bcc0-7c0f890fede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:00.677834 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d6ace-19be-4580-bcc0-7c0f890fede4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:00.678363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d6ace-19be-4580-bcc0-7c0f890fede4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:00.678865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d6ace-19be-4580-bcc0-7c0f890fede4 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:00.679758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d6ace-19be-4580-bcc0-7c0f890fede4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:00.680314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d6ace-19be-4580-bcc0-7c0f890fede4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:00.681910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d6ace-19be-4580-bcc0-7c0f890fede4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:00.690957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 239/490] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:00.697308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d6ace-19be-4580-bcc0-7c0f890fede4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:00.808588 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152b05c-e335-4e73-a40e-8064fadfbfe7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:00.825316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d6ace-19be-4580-bcc0-7c0f890fede4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:00.843481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d94ed-bf95-40cf-9610-46c40a8cf766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:00.843860 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d94ed-bf95-40cf-9610-46c40a8cf766 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:00.844142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d94ed-bf95-40cf-9610-46c40a8cf766 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:00.844418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d94ed-bf95-40cf-9610-46c40a8cf766 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:00.852275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d94ed-bf95-40cf-9610-46c40a8cf766 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:00.853686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d94ed-bf95-40cf-9610-46c40a8cf766 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:00.854446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:00.863776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d3fde-bba6-40a6-901d-859d1790d041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:00.865411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d3fde-bba6-40a6-901d-859d1790d041 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:00.867591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d3fde-bba6-40a6-901d-859d1790d041 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:00.867591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d3fde-bba6-40a6-901d-859d1790d041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0e1d94ed-bf95-40cf-9610-46c40a8cf766 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:00.868511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:00.869418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:00.870324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:00.870324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:00.870324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:00.870324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:00.870324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:00.870324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:00.870324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:00.870324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:00.870324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:00.870324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:00.870324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 252/491] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Aug 11 09:41:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:00.871340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8046d-86ad-48fc-9a6c-c5d0d9ce0713 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:00.871507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8046d-86ad-48fc-9a6c-c5d0d9ce0713 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:00.871762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8046d-86ad-48fc-9a6c-c5d0d9ce0713 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:00.877077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 253/492] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:00.877888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8046d-86ad-48fc-9a6c-c5d0d9ce0713 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:00.882917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d3fde-bba6-40a6-901d-859d1790d041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:00.884667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d3fde-bba6-40a6-901d-859d1790d041 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-469d3fde-bba6-40a6-901d-859d1790d041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:00.901594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:00.902538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:00.903357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:00.903357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:00.903357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:00.905515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 240/493] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:00.907141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac71c1b-b7af-4229-a2bf-1efd4b34ef98 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:00.907644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac71c1b-b7af-4229-a2bf-1efd4b34ef98 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:00.908114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac71c1b-b7af-4229-a2bf-1efd4b34ef98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:00.929494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6ac71c1b-b7af-4229-a2bf-1efd4b34ef98 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:00.931296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6ac71c1b-b7af-4229-a2bf-1efd4b34ef98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:00.950834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:00.991205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8046d-86ad-48fc-9a6c-c5d0d9ce0713 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:01.027486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 241/494] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:01.028661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac71c1b-b7af-4229-a2bf-1efd4b34ef98 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:01.037767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8799f8-3694-4f17-9eb7-2b3a3a6ef2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:01.038813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8799f8-3694-4f17-9eb7-2b3a3a6ef2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.038813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8799f8-3694-4f17-9eb7-2b3a3a6ef2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.039008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8799f8-3694-4f17-9eb7-2b3a3a6ef2e5 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.039720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8799f8-3694-4f17-9eb7-2b3a3a6ef2e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:01.040032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8799f8-3694-4f17-9eb7-2b3a3a6ef2e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:01.041731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8799f8-3694-4f17-9eb7-2b3a3a6ef2e5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:01.094514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 254/495] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Aug 11 09:41:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:01.095293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8799f8-3694-4f17-9eb7-2b3a3a6ef2e5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:01.148165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac71c1b-b7af-4229-a2bf-1efd4b34ef98 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:01.159396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:01.195113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34ca67-422e-4486-9977-e59745cd6ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:01.195738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34ca67-422e-4486-9977-e59745cd6ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.196016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34ca67-422e-4486-9977-e59745cd6ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.196285 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f34ca67-422e-4486-9977-e59745cd6ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.196962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34ca67-422e-4486-9977-e59745cd6ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:01.197289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34ca67-422e-4486-9977-e59745cd6ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:01.198678 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f34ca67-422e-4486-9977-e59745cd6ac6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:01.214809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 242/496] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:01.218268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34ca67-422e-4486-9977-e59745cd6ac6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:01.246267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8799f8-3694-4f17-9eb7-2b3a3a6ef2e5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:01.274043 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:01.283200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df094d3-1f06-4728-93d4-1408901340fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:01.288642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df094d3-1f06-4728-93d4-1408901340fe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.288642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df094d3-1f06-4728-93d4-1408901340fe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.288642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df094d3-1f06-4728-93d4-1408901340fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.294968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df094d3-1f06-4728-93d4-1408901340fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6df094d3-1f06-4728-93d4-1408901340fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:01.299896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:01.300750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 255/497] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:01.307231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf68054-1ee4-4dcd-a40d-5a3d244322e3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.307932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf68054-1ee4-4dcd-a40d-5a3d244322e3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.307932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf68054-1ee4-4dcd-a40d-5a3d244322e3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.307932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 256/498] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:01.307932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf68054-1ee4-4dcd-a40d-5a3d244322e3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:01.340501 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f34ca67-422e-4486-9977-e59745cd6ac6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:01.380611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13651d17-cc0c-4530-a20d-9410a7af8bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:01.380611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13651d17-cc0c-4530-a20d-9410a7af8bda None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.380611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13651d17-cc0c-4530-a20d-9410a7af8bda None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.380611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13651d17-cc0c-4530-a20d-9410a7af8bda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.418853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:01.419286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf68054-1ee4-4dcd-a40d-5a3d244322e3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:01.455531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13651d17-cc0c-4530-a20d-9410a7af8bda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:01.456101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2341-98c6-4f9e-99d9-ab240afe725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:01.456839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2341-98c6-4f9e-99d9-ab240afe725f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.456986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13651d17-cc0c-4530-a20d-9410a7af8bda None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:01.457332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2341-98c6-4f9e-99d9-ab240afe725f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.457707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a2341-98c6-4f9e-99d9-ab240afe725f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.465376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a2341-98c6-4f9e-99d9-ab240afe725f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:01.466891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a2341-98c6-4f9e-99d9-ab240afe725f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-699a2341-98c6-4f9e-99d9-ab240afe725f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:01.479136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:01.480228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 257/499] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:01.486087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2341-98c6-4f9e-99d9-ab240afe725f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:01.530515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:01.568710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 243/500] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:01.569931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05ccc0-d57a-4bae-b070-4a2db6ac1fe4 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.570215 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05ccc0-d57a-4bae-b070-4a2db6ac1fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.570488 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05ccc0-d57a-4bae-b070-4a2db6ac1fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.586888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a2341-98c6-4f9e-99d9-ab240afe725f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:01.587378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0b05ccc0-d57a-4bae-b070-4a2db6ac1fe4 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:01.588443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0b05ccc0-d57a-4bae-b070-4a2db6ac1fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:01.621552 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6de27c-35bd-4ffa-94a6-988eaa888580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:01.622159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6de27c-35bd-4ffa-94a6-988eaa888580 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.622441 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6de27c-35bd-4ffa-94a6-988eaa888580 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.622713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6de27c-35bd-4ffa-94a6-988eaa888580 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.623377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6de27c-35bd-4ffa-94a6-988eaa888580 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:01.623682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6de27c-35bd-4ffa-94a6-988eaa888580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:01.625068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6de27c-35bd-4ffa-94a6-988eaa888580 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:01.633283 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 258/501] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:01.634017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6de27c-35bd-4ffa-94a6-988eaa888580 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:01.649332 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:01.654622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 244/502] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:01.655838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfe3ce-e54c-447e-a21b-fdfc02bdada2 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.656123 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfe3ce-e54c-447e-a21b-fdfc02bdada2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.656403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfe3ce-e54c-447e-a21b-fdfc02bdada2 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.657699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 245/503] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:01.658392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfe3ce-e54c-447e-a21b-fdfc02bdada2 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:01.741132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6de27c-35bd-4ffa-94a6-988eaa888580 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:01.749670 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:01.760028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfe3ce-e54c-447e-a21b-fdfc02bdada2 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:01.773966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a825acb-e7cc-4930-b307-a3c2de6d1417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:01.774590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a825acb-e7cc-4930-b307-a3c2de6d1417 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.774876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a825acb-e7cc-4930-b307-a3c2de6d1417 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.775150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a825acb-e7cc-4930-b307-a3c2de6d1417 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.775883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a825acb-e7cc-4930-b307-a3c2de6d1417 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:01.776255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a825acb-e7cc-4930-b307-a3c2de6d1417 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:01.778388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a825acb-e7cc-4930-b307-a3c2de6d1417 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:01.794456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1a7b5-16fa-4466-8c80-e0bbbf963398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:01.795079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1a7b5-16fa-4466-8c80-e0bbbf963398 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.795360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1a7b5-16fa-4466-8c80-e0bbbf963398 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.795633 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1a7b5-16fa-4466-8c80-e0bbbf963398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.796391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 259/504] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Aug 11 09:41:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:01.797227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a825acb-e7cc-4930-b307-a3c2de6d1417 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:01.810315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1a7b5-16fa-4466-8c80-e0bbbf963398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:01.811785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1a7b5-16fa-4466-8c80-e0bbbf963398 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-6fc1a7b5-16fa-4466-8c80-e0bbbf963398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:01.826267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:01.827218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:01.828244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:01.828244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:01.828244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:01.828244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 246/505] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:01.828556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748293da-9e87-4b32-a0f2-63a6fac753d7 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.828751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748293da-9e87-4b32-a0f2-63a6fac753d7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.829043 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748293da-9e87-4b32-a0f2-63a6fac753d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.846535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-748293da-9e87-4b32-a0f2-63a6fac753d7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:01.847729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-748293da-9e87-4b32-a0f2-63a6fac753d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:01.893101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:01.899667 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a825acb-e7cc-4930-b307-a3c2de6d1417 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:01.914215 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 247/506] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:01.915043 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748293da-9e87-4b32-a0f2-63a6fac753d7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:01.942765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b98fd5-56c7-4a70-8235-f969a7c2c17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:01.943489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b98fd5-56c7-4a70-8235-f969a7c2c17c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:01.943753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b98fd5-56c7-4a70-8235-f969a7c2c17c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:01.944028 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b98fd5-56c7-4a70-8235-f969a7c2c17c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:01.955065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b98fd5-56c7-4a70-8235-f969a7c2c17c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:01.956490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b98fd5-56c7-4a70-8235-f969a7c2c17c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b2b98fd5-56c7-4a70-8235-f969a7c2c17c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:01.971356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:01.972431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:01.973418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:01.973418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:01.973418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:01.973418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 260/507] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:01.973418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b98fd5-56c7-4a70-8235-f969a7c2c17c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:01.992298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:02.017921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748293da-9e87-4b32-a0f2-63a6fac753d7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.051355 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafad93-18f4-4609-a4dd-13362fdb20d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:02.052004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafad93-18f4-4609-a4dd-13362fdb20d0 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:02.052309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafad93-18f4-4609-a4dd-13362fdb20d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:02.052628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafad93-18f4-4609-a4dd-13362fdb20d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:02.062429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafad93-18f4-4609-a4dd-13362fdb20d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-7eafad93-18f4-4609-a4dd-13362fdb20d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:02.067368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:02.068359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:02.069153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 248/508] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:02.069773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafad93-18f4-4609-a4dd-13362fdb20d0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:02.077221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b98fd5-56c7-4a70-8235-f969a7c2c17c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.111212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b73f-dd43-4c43-9dea-aeaaea852ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:02.111894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b73f-dd43-4c43-9dea-aeaaea852ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:02.112175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b73f-dd43-4c43-9dea-aeaaea852ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:02.112445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b73f-dd43-4c43-9dea-aeaaea852ac7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:02.113480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:02.171571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafad93-18f4-4609-a4dd-13362fdb20d0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.180087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961b73f-dd43-4c43-9dea-aeaaea852ac7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:02.181583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961b73f-dd43-4c43-9dea-aeaaea852ac7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:02.200660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:02.211284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b2f58-4bd3-43ff-9075-0c6557381cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:02.211890 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b2f58-4bd3-43ff-9075-0c6557381cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:02.218175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b2f58-4bd3-43ff-9075-0c6557381cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:02.218175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b2f58-4bd3-43ff-9075-0c6557381cf8 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:02.218175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b2f58-4bd3-43ff-9075-0c6557381cf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:02.218175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b2f58-4bd3-43ff-9075-0c6557381cf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:02.218175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b2f58-4bd3-43ff-9075-0c6557381cf8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:02.226076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 249/509] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:02.230643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b2f58-4bd3-43ff-9075-0c6557381cf8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:02.281641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 261/510] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:02.282480 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b73f-dd43-4c43-9dea-aeaaea852ac7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:02.322884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:02.325794 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b2f58-4bd3-43ff-9075-0c6557381cf8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.359278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba6705-29d6-41ea-a272-a3556750a3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:02.359894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba6705-29d6-41ea-a272-a3556750a3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:02.360177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba6705-29d6-41ea-a272-a3556750a3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:02.360446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba6705-29d6-41ea-a272-a3556750a3f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:02.370958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba6705-29d6-41ea-a272-a3556750a3f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:02.372621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba6705-29d6-41ea-a272-a3556750a3f2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:02.380967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b73f-dd43-4c43-9dea-aeaaea852ac7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-87ba6705-29d6-41ea-a272-a3556750a3f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:02.387038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:02.388008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:02.394274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:02.394274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:02.394274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:02.394274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 250/511] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:02.394274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba6705-29d6-41ea-a272-a3556750a3f2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:02.398475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:02.414564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef1269-809c-4d4c-8fd9-c2a52902a862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:02.415165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef1269-809c-4d4c-8fd9-c2a52902a862 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:02.415443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef1269-809c-4d4c-8fd9-c2a52902a862 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:02.415710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef1269-809c-4d4c-8fd9-c2a52902a862 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:02.416379 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef1269-809c-4d4c-8fd9-c2a52902a862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:02.416676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef1269-809c-4d4c-8fd9-c2a52902a862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:02.418143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef1269-809c-4d4c-8fd9-c2a52902a862 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:02.430084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 262/512] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:02.430804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef1269-809c-4d4c-8fd9-c2a52902a862 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:02.493887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba6705-29d6-41ea-a272-a3556750a3f2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.526889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4196f01-3e19-48eb-960f-81887fed5461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:02.527494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196f01-3e19-48eb-960f-81887fed5461 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:02.527780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196f01-3e19-48eb-960f-81887fed5461 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:02.528058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196f01-3e19-48eb-960f-81887fed5461 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:02.528728 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4196f01-3e19-48eb-960f-81887fed5461 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:02.529053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4196f01-3e19-48eb-960f-81887fed5461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:02.530436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4196f01-3e19-48eb-960f-81887fed5461 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:02.532223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef1269-809c-4d4c-8fd9-c2a52902a862 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.538411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 251/513] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:02.539210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4196f01-3e19-48eb-960f-81887fed5461 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:02.555204 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:02.569959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a07d63-5632-4584-aac6-0a515f38428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:02.570594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a07d63-5632-4584-aac6-0a515f38428c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:02.570874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a07d63-5632-4584-aac6-0a515f38428c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:02.571149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a07d63-5632-4584-aac6-0a515f38428c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:02.577901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a07d63-5632-4584-aac6-0a515f38428c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:02.579328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a07d63-5632-4584-aac6-0a515f38428c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b7a07d63-5632-4584-aac6-0a515f38428c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:02.593215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:02.594592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:02.595407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:02.595407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:02.595407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:02.595407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:02.595407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:02.595407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:02.595407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:02.595407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:02.595407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 263/514] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:02.595407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a07d63-5632-4584-aac6-0a515f38428c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:02.633728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:02.644715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4196f01-3e19-48eb-960f-81887fed5461 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.678447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071cfa8-ba6e-424a-85a0-039de975c017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:02.679064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071cfa8-ba6e-424a-85a0-039de975c017 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:02.679340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071cfa8-ba6e-424a-85a0-039de975c017 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:02.679609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071cfa8-ba6e-424a-85a0-039de975c017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:02.689804 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071cfa8-ba6e-424a-85a0-039de975c017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:02.691166 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071cfa8-ba6e-424a-85a0-039de975c017 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:02.695470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a07d63-5632-4584-aac6-0a515f38428c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-5071cfa8-ba6e-424a-85a0-039de975c017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:02.704850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:02.705771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:02.706587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:02.706587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:02.706587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:02.706587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 252/515] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:02.706795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071cfa8-ba6e-424a-85a0-039de975c017 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:02.731884 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99697c08-aae0-4aca-8a7f-f061ecf97cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:02.732492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99697c08-aae0-4aca-8a7f-f061ecf97cbb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:02.732772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99697c08-aae0-4aca-8a7f-f061ecf97cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:02.733055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99697c08-aae0-4aca-8a7f-f061ecf97cbb None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:02.733697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99697c08-aae0-4aca-8a7f-f061ecf97cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:02.733994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99697c08-aae0-4aca-8a7f-f061ecf97cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:02.735341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99697c08-aae0-4aca-8a7f-f061ecf97cbb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:02.746210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 264/516] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:02.746992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99697c08-aae0-4aca-8a7f-f061ecf97cbb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:02.776138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:02.812564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071cfa8-ba6e-424a-85a0-039de975c017 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.841806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:02.845258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99697c08-aae0-4aca-8a7f-f061ecf97cbb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.845465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136db48-c5b8-47ab-84f8-3378e0a447a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:02.846080 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136db48-c5b8-47ab-84f8-3378e0a447a8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:02.846354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136db48-c5b8-47ab-84f8-3378e0a447a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:02.846629 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136db48-c5b8-47ab-84f8-3378e0a447a8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:02.853901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136db48-c5b8-47ab-84f8-3378e0a447a8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:02.858394 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136db48-c5b8-47ab-84f8-3378e0a447a8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-4136db48-c5b8-47ab-84f8-3378e0a447a8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:02.868021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:02.869119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 253/517] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:02.870243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136db48-c5b8-47ab-84f8-3378e0a447a8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:02.885468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a674a06-8261-43dd-8d80-752d32ee135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:02.886288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a674a06-8261-43dd-8d80-752d32ee135e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:02.887746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a674a06-8261-43dd-8d80-752d32ee135e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:02.888021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a674a06-8261-43dd-8d80-752d32ee135e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:02.898688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a674a06-8261-43dd-8d80-752d32ee135e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:02.900121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a674a06-8261-43dd-8d80-752d32ee135e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6a674a06-8261-43dd-8d80-752d32ee135e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:02.914769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:02.915686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:02.917129 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:02.917129 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:02.917129 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:02.917129 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 265/518] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:02.917129 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a674a06-8261-43dd-8d80-752d32ee135e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:02.971504 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136db48-c5b8-47ab-84f8-3378e0a447a8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:02.995701 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:03.004078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c1812-3bfa-465a-bf3e-0f0a8997a636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:03.004676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c1812-3bfa-465a-bf3e-0f0a8997a636 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:03.006952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c1812-3bfa-465a-bf3e-0f0a8997a636 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:03.006952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c1812-3bfa-465a-bf3e-0f0a8997a636 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:03.006952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c1812-3bfa-465a-bf3e-0f0a8997a636 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:03.006952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c1812-3bfa-465a-bf3e-0f0a8997a636 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:03.008413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c1812-3bfa-465a-bf3e-0f0a8997a636 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:03.015845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a674a06-8261-43dd-8d80-752d32ee135e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:03.025883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 254/519] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:03.025883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c1812-3bfa-465a-bf3e-0f0a8997a636 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:03.054534 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d77c6-4843-4b87-bc6a-6c4c2cfd1392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:03.055143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d77c6-4843-4b87-bc6a-6c4c2cfd1392 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:03.055424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d77c6-4843-4b87-bc6a-6c4c2cfd1392 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:03.055694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d77c6-4843-4b87-bc6a-6c4c2cfd1392 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:03.056336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d77c6-4843-4b87-bc6a-6c4c2cfd1392 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:03.056639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d77c6-4843-4b87-bc6a-6c4c2cfd1392 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:03.058026 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d77c6-4843-4b87-bc6a-6c4c2cfd1392 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:03.070464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 266/520] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:03.071262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d77c6-4843-4b87-bc6a-6c4c2cfd1392 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:03.083747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:03.124687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c1812-3bfa-465a-bf3e-0f0a8997a636 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:03.158987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cca9cf-2f06-4e7f-ac5d-6bb4e970177c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:03.159825 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cca9cf-2f06-4e7f-ac5d-6bb4e970177c None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:03.160104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cca9cf-2f06-4e7f-ac5d-6bb4e970177c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:03.160376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cca9cf-2f06-4e7f-ac5d-6bb4e970177c None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:03.161050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cca9cf-2f06-4e7f-ac5d-6bb4e970177c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:03.162699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cca9cf-2f06-4e7f-ac5d-6bb4e970177c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:03.170464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d77c6-4843-4b87-bc6a-6c4c2cfd1392 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:03.204120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0010f-ddcc-4b7d-b594-f0ad823d4f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:03.204774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0010f-ddcc-4b7d-b594-f0ad823d4f9d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:03.205087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0010f-ddcc-4b7d-b594-f0ad823d4f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:03.205362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0010f-ddcc-4b7d-b594-f0ad823d4f9d None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:03.206084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0010f-ddcc-4b7d-b594-f0ad823d4f9d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:03.206385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0010f-ddcc-4b7d-b594-f0ad823d4f9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:03.207785 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0010f-ddcc-4b7d-b594-f0ad823d4f9d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:03.229071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 267/521] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Aug 11 09:41:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:03.229071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0010f-ddcc-4b7d-b594-f0ad823d4f9d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:03.258234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 255/522] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:41:03 2022] POST /identity/v3/users => generated 247 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:03.259065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cca9cf-2f06-4e7f-ac5d-6bb4e970177c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:03.296291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:03.318779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:03.336921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0010f-ddcc-4b7d-b594-f0ad823d4f9d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:03.377237 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2953953-0031-4113-89d2-71f449c6e9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:03.377870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2953953-0031-4113-89d2-71f449c6e9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:03.378153 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2953953-0031-4113-89d2-71f449c6e9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:03.378427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2953953-0031-4113-89d2-71f449c6e9a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:03.379098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2953953-0031-4113-89d2-71f449c6e9a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:03.379407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2953953-0031-4113-89d2-71f449c6e9a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:03.379711 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cca9cf-2f06-4e7f-ac5d-6bb4e970177c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:03.380868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2953953-0031-4113-89d2-71f449c6e9a3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:03.395277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 268/523] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:03.396588 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2953953-0031-4113-89d2-71f449c6e9a3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:03.428257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e3088-c92d-43b8-8fae-ff5b0521d01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:03.428873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e3088-c92d-43b8-8fae-ff5b0521d01b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:03.429201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e3088-c92d-43b8-8fae-ff5b0521d01b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:03.429483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e3088-c92d-43b8-8fae-ff5b0521d01b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:03.430961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e3088-c92d-43b8-8fae-ff5b0521d01b None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=34fa13a87ebe4b789ca1884e5aeb9237) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:03.430961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e3088-c92d-43b8-8fae-ff5b0521d01b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:03.432568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e3088-c92d-43b8-8fae-ff5b0521d01b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:03.437940 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 256/524] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:03 2022] GET /identity/v3/role_assignments?user.id=34fa13a87ebe4b789ca1884e5aeb9237&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:03.443562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e3088-c92d-43b8-8fae-ff5b0521d01b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:03.525634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2953953-0031-4113-89d2-71f449c6e9a3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:03.555855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:03.564337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616161a6-9f51-4a22-9f85-12f094daf357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:03.565208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616161a6-9f51-4a22-9f85-12f094daf357 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:03.565590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616161a6-9f51-4a22-9f85-12f094daf357 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:03.565948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616161a6-9f51-4a22-9f85-12f094daf357 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:03.572848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e3088-c92d-43b8-8fae-ff5b0521d01b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:03.575426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616161a6-9f51-4a22-9f85-12f094daf357 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:03.577049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616161a6-9f51-4a22-9f85-12f094daf357 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:03.578915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-616161a6-9f51-4a22-9f85-12f094daf357 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:03.590112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:03.591099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:03.591904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:03.591904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:03.591904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:03.591904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:03.591904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:03.591904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:03.591904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:03.591904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:03.591904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 269/525] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:03.592346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616161a6-9f51-4a22-9f85-12f094daf357 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:03.620255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d1941-08b7-4555-9df9-fffb87de29ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:03.621413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d1941-08b7-4555-9df9-fffb87de29ef None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:03.622183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d1941-08b7-4555-9df9-fffb87de29ef None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:03.622663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d1941-08b7-4555-9df9-fffb87de29ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:03.713702 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d1941-08b7-4555-9df9-fffb87de29ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:03.715179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654d1941-08b7-4555-9df9-fffb87de29ef None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:03.724902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616161a6-9f51-4a22-9f85-12f094daf357 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:03.763633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd8070-ceae-4798-9deb-81750886e841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:03.764635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd8070-ceae-4798-9deb-81750886e841 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:03.765156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd8070-ceae-4798-9deb-81750886e841 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:03.765707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd8070-ceae-4798-9deb-81750886e841 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:03.774850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd8070-ceae-4798-9deb-81750886e841 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:03.776873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd8070-ceae-4798-9deb-81750886e841 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-addd8070-ceae-4798-9deb-81750886e841 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:03.792086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:03.793138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:03.793992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:03.793992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:03.793992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:03.793992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:03.793992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:03.793992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:03.793992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:03.793992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:03.793992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 09:41:03.793992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:03.796929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 270/526] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:03.798420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd8070-ceae-4798-9deb-81750886e841 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:03.828443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 257/527] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:03.828443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d1941-08b7-4555-9df9-fffb87de29ef None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:03.869768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:03.877392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:03.931621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d1941-08b7-4555-9df9-fffb87de29ef None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:03.934647 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd8070-ceae-4798-9deb-81750886e841 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:03.967391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca4e3a-8c74-4dd3-9677-293fcb4ce5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:03.968006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca4e3a-8c74-4dd3-9677-293fcb4ce5da None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:03.968286 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca4e3a-8c74-4dd3-9677-293fcb4ce5da None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:03.968560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca4e3a-8c74-4dd3-9677-293fcb4ce5da None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:03.969350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca4e3a-8c74-4dd3-9677-293fcb4ce5da None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:03.969660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca4e3a-8c74-4dd3-9677-293fcb4ce5da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:03.971088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca4e3a-8c74-4dd3-9677-293fcb4ce5da None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:03.972249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257213e-cf52-4255-9e63-b8dc05e666d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:03.973094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257213e-cf52-4255-9e63-b8dc05e666d3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:03.973595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257213e-cf52-4255-9e63-b8dc05e666d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:03.974057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257213e-cf52-4255-9e63-b8dc05e666d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:03.974966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257213e-cf52-4255-9e63-b8dc05e666d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:03.975477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257213e-cf52-4255-9e63-b8dc05e666d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:03.977108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257213e-cf52-4255-9e63-b8dc05e666d3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:03.995801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 271/528] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:03.996950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257213e-cf52-4255-9e63-b8dc05e666d3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:04.011619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 258/529] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Aug 11 09:41:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:04.012698 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca4e3a-8c74-4dd3-9677-293fcb4ce5da None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:04.093462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:04.101890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257213e-cf52-4255-9e63-b8dc05e666d3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:04.112156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:04.124198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca4e3a-8c74-4dd3-9677-293fcb4ce5da None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:04.138199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1d95a-7b5c-4d77-9c61-897a49163f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:04.138833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d95a-7b5c-4d77-9c61-897a49163f5f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:04.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d95a-7b5c-4d77-9c61-897a49163f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:04.139384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d95a-7b5c-4d77-9c61-897a49163f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:04.153592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1d95a-7b5c-4d77-9c61-897a49163f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:04.155032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1d95a-7b5c-4d77-9c61-897a49163f5f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e2b1d95a-7b5c-4d77-9c61-897a49163f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:04.173462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:04.174372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:04.175183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:04.175183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:04.175183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:04.177523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90440aa-9aa3-43e3-847c-13b03b5e1821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:04.178413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90440aa-9aa3-43e3-847c-13b03b5e1821 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:04.178983 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90440aa-9aa3-43e3-847c-13b03b5e1821 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:04.179250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 272/530] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:04.179709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90440aa-9aa3-43e3-847c-13b03b5e1821 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:04.180089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1d95a-7b5c-4d77-9c61-897a49163f5f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:04.181121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90440aa-9aa3-43e3-847c-13b03b5e1821 None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=d83448e4c6704f7c8771132bd67eaa1d) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:04.181634 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90440aa-9aa3-43e3-847c-13b03b5e1821 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:04.183333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90440aa-9aa3-43e3-847c-13b03b5e1821 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:04.191604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 259/531] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:04 2022] GET /identity/v3/role_assignments?user.id=d83448e4c6704f7c8771132bd67eaa1d&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:04.192958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90440aa-9aa3-43e3-847c-13b03b5e1821 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:04.320010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1d95a-7b5c-4d77-9c61-897a49163f5f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:04.370316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00899022-5b64-4181-b209-9bdc6ab9b82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:04.371306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899022-5b64-4181-b209-9bdc6ab9b82f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:04.371788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899022-5b64-4181-b209-9bdc6ab9b82f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:04.372261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00899022-5b64-4181-b209-9bdc6ab9b82f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:04.398723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90440aa-9aa3-43e3-847c-13b03b5e1821 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:04.422562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:04.448874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:04.463413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2e311-9c4a-4988-bc94-3fdd5375e113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:04.465265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e311-9c4a-4988-bc94-3fdd5375e113 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:04.465918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e311-9c4a-4988-bc94-3fdd5375e113 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:04.466521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e311-9c4a-4988-bc94-3fdd5375e113 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:04.468608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2e311-9c4a-4988-bc94-3fdd5375e113 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:04.480541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2e311-9c4a-4988-bc94-3fdd5375e113 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:04.480541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2e311-9c4a-4988-bc94-3fdd5375e113 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:04.498960 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 260/532] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:04.505259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2e311-9c4a-4988-bc94-3fdd5375e113 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:04.515287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00899022-5b64-4181-b209-9bdc6ab9b82f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:04.516758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00899022-5b64-4181-b209-9bdc6ab9b82f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:04.652142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2e311-9c4a-4988-bc94-3fdd5375e113 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:04.702672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 273/533] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:04.704982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00899022-5b64-4181-b209-9bdc6ab9b82f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:04.734436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1a254-f893-452b-81c4-3ca2722caac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:04.735767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1a254-f893-452b-81c4-3ca2722caac6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:04.736250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1a254-f893-452b-81c4-3ca2722caac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:04.736762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1a254-f893-452b-81c4-3ca2722caac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:04.752919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1a254-f893-452b-81c4-3ca2722caac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:04.754720 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1a254-f893-452b-81c4-3ca2722caac6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-aae1a254-f893-452b-81c4-3ca2722caac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:04.771228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:04.772165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:04.773003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:04.773003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:04.773003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:04.773003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:04.773003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:04.773003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:04.773003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:04.773003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:04.775486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 261/534] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:04.782197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1a254-f893-452b-81c4-3ca2722caac6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:04.819497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:04.863071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:04.910630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00899022-5b64-4181-b209-9bdc6ab9b82f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:04.943547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1a254-f893-452b-81c4-3ca2722caac6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:04.964555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d38d6-7381-4899-a4a0-6906d8ceed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:04.965230 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d38d6-7381-4899-a4a0-6906d8ceed4d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:04.965515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d38d6-7381-4899-a4a0-6906d8ceed4d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:04.965788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d38d6-7381-4899-a4a0-6906d8ceed4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:04.979360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d38d6-7381-4899-a4a0-6906d8ceed4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:04.980809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d38d6-7381-4899-a4a0-6906d8ceed4d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-997d38d6-7381-4899-a4a0-6906d8ceed4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:05.005257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:05.006191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:05.007010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:05.007010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:05.007010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:05.007010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 274/535] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:05.010798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8ebee-7319-4b3b-afab-8034bf2298d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:05.010798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8ebee-7319-4b3b-afab-8034bf2298d4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:05.010798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8ebee-7319-4b3b-afab-8034bf2298d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:05.010798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8ebee-7319-4b3b-afab-8034bf2298d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:05.011134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8ebee-7319-4b3b-afab-8034bf2298d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:05.011134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8ebee-7319-4b3b-afab-8034bf2298d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:05.011794 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a8ebee-7319-4b3b-afab-8034bf2298d4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:05.014413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d38d6-7381-4899-a4a0-6906d8ceed4d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:05.027336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 262/536] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:05.032917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8ebee-7319-4b3b-afab-8034bf2298d4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:05.171837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8ebee-7319-4b3b-afab-8034bf2298d4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:05.185227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:05.235684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b5137-f9de-498d-9dc1-e34f076fe634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:05.236536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b5137-f9de-498d-9dc1-e34f076fe634 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:05.236996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b5137-f9de-498d-9dc1-e34f076fe634 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:05.237425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b5137-f9de-498d-9dc1-e34f076fe634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:05.238291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b5137-f9de-498d-9dc1-e34f076fe634 None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=029bb992f9974eec83c0fe7f9fa58826) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:05.238734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b5137-f9de-498d-9dc1-e34f076fe634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:05.239847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d38d6-7381-4899-a4a0-6906d8ceed4d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:05.240573 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b5137-f9de-498d-9dc1-e34f076fe634 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:05.253638 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 263/537] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:05 2022] GET /identity/v3/role_assignments?user.id=029bb992f9974eec83c0fe7f9fa58826&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:05.254463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:05.279260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879237c3-789b-48d4-82bc-d4f82b1d334c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:05.279896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879237c3-789b-48d4-82bc-d4f82b1d334c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:05.280178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879237c3-789b-48d4-82bc-d4f82b1d334c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:05.280453 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879237c3-789b-48d4-82bc-d4f82b1d334c None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:05.281172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879237c3-789b-48d4-82bc-d4f82b1d334c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:05.281486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879237c3-789b-48d4-82bc-d4f82b1d334c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:05.282882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879237c3-789b-48d4-82bc-d4f82b1d334c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:05.297188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 275/538] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:05.302392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b5137-f9de-498d-9dc1-e34f076fe634 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:05.458291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b5137-f9de-498d-9dc1-e34f076fe634 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:05.494800 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972ee31-8579-4cda-acc4-134b551bd29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:05.495732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972ee31-8579-4cda-acc4-134b551bd29e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:05.496270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972ee31-8579-4cda-acc4-134b551bd29e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:05.496722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972ee31-8579-4cda-acc4-134b551bd29e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:05.504048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972ee31-8579-4cda-acc4-134b551bd29e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:05.506020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972ee31-8579-4cda-acc4-134b551bd29e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a972ee31-8579-4cda-acc4-134b551bd29e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:05.519193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:05.520089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:05.521705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:05.521705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:05.521705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:05.521705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:05.521705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:05.521705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:05.521705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:05.521705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:05.524022 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 264/539] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:05.529445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879237c3-789b-48d4-82bc-d4f82b1d334c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:05.605778 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22268550-1ed8-42e7-adc9-e5971177c4b2 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:05.606469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22268550-1ed8-42e7-adc9-e5971177c4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:05.606897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22268550-1ed8-42e7-adc9-e5971177c4b2 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:05.609518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 265/540] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:05.614359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f570393-e97d-4359-97da-f9d860e4beb2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:05.614848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f570393-e97d-4359-97da-f9d860e4beb2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:05.615086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f570393-e97d-4359-97da-f9d860e4beb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:05.633827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9f570393-e97d-4359-97da-f9d860e4beb2 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:05.635040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9f570393-e97d-4359-97da-f9d860e4beb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:05.662967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:05.667735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879237c3-789b-48d4-82bc-d4f82b1d334c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:05.709678 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce7c45-2424-4951-90b1-e81ec4499a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:05.710905 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce7c45-2424-4951-90b1-e81ec4499a87 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:05.711772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce7c45-2424-4951-90b1-e81ec4499a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:05.712253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce7c45-2424-4951-90b1-e81ec4499a87 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:05.713200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce7c45-2424-4951-90b1-e81ec4499a87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:05.713707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce7c45-2424-4951-90b1-e81ec4499a87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:05.716219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce7c45-2424-4951-90b1-e81ec4499a87 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:05.732610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 266/541] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:41:05.733614 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 276/542] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:05.742947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f570393-e97d-4359-97da-f9d860e4beb2 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:05.765098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0251d4b-bc73-409a-a52e-d62a6d6c8af9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:05.765392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0251d4b-bc73-409a-a52e-d62a6d6c8af9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:05.765664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0251d4b-bc73-409a-a52e-d62a6d6c8af9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:05.767256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 277/543] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:05.772957 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62c1b4-b11e-4c81-afee-eda365ca6b07 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:05.773326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62c1b4-b11e-4c81-afee-eda365ca6b07 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:05.773783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62c1b4-b11e-4c81-afee-eda365ca6b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:05.798633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8b62c1b4-b11e-4c81-afee-eda365ca6b07 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:05.800174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8b62c1b4-b11e-4c81-afee-eda365ca6b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:05.834062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:05.860079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f570393-e97d-4359-97da-f9d860e4beb2 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:05.890467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 278/544] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:41:05.892320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62c1b4-b11e-4c81-afee-eda365ca6b07 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:05.897798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274e612-fdaa-4840-8f49-d2348c6483b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:05.898500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274e612-fdaa-4840-8f49-d2348c6483b2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:05.898782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274e612-fdaa-4840-8f49-d2348c6483b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:05.899054 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274e612-fdaa-4840-8f49-d2348c6483b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:05.899788 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274e612-fdaa-4840-8f49-d2348c6483b2 None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=4fc6c3d1ce8645808c22d1b59b15a100) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:05.900090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274e612-fdaa-4840-8f49-d2348c6483b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:05.901636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274e612-fdaa-4840-8f49-d2348c6483b2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:05.908135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 267/545] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:05 2022] GET /identity/v3/role_assignments?user.id=4fc6c3d1ce8645808c22d1b59b15a100&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:05.925300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274e612-fdaa-4840-8f49-d2348c6483b2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:05.972726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:06.064953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62c1b4-b11e-4c81-afee-eda365ca6b07 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:06.121037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274e612-fdaa-4840-8f49-d2348c6483b2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:06.148719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77344e59-3fe8-443f-9bc2-83fdb3c1f9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:06.149590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77344e59-3fe8-443f-9bc2-83fdb3c1f9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:06.149893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77344e59-3fe8-443f-9bc2-83fdb3c1f9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:06.150173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77344e59-3fe8-443f-9bc2-83fdb3c1f9c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:06.150677 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e2723-9e3c-4070-baec-1a9b89edae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:06.151310 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e2723-9e3c-4070-baec-1a9b89edae09 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:06.151585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e2723-9e3c-4070-baec-1a9b89edae09 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:06.151856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e2723-9e3c-4070-baec-1a9b89edae09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:06.171034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e2723-9e3c-4070-baec-1a9b89edae09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:06.177356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77344e59-3fe8-443f-9bc2-83fdb3c1f9c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-77344e59-3fe8-443f-9bc2-83fdb3c1f9c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:06.182123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:06.182985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:06.183766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-3b2e2723-9e3c-4070-baec-1a9b89edae09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:06.184342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:06.185948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:06.186754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:06.187917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 279/547] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:06.188001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 268/547] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:06.194567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77344e59-3fe8-443f-9bc2-83fdb3c1f9c0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:06.195913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e2723-9e3c-4070-baec-1a9b89edae09 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:06.246607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:06.332914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e2723-9e3c-4070-baec-1a9b89edae09 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:06.341368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:06.370515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea19ff4-3c2c-41f0-bdfc-5763a37a6834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:06.371124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea19ff4-3c2c-41f0-bdfc-5763a37a6834 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:06.374078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea19ff4-3c2c-41f0-bdfc-5763a37a6834 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:06.374078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea19ff4-3c2c-41f0-bdfc-5763a37a6834 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:06.374078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea19ff4-3c2c-41f0-bdfc-5763a37a6834 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:06.374078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea19ff4-3c2c-41f0-bdfc-5763a37a6834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:06.374395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea19ff4-3c2c-41f0-bdfc-5763a37a6834 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:06.384958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 269/548] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:06.391306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea19ff4-3c2c-41f0-bdfc-5763a37a6834 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:06.498506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77344e59-3fe8-443f-9bc2-83fdb3c1f9c0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:06.502375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea19ff4-3c2c-41f0-bdfc-5763a37a6834 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:06.540653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ac50d-ab4d-4d23-9106-d6ed884aaba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:06.541399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac50d-ab4d-4d23-9106-d6ed884aaba5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:06.541685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac50d-ab4d-4d23-9106-d6ed884aaba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:06.541965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ac50d-ab4d-4d23-9106-d6ed884aaba5 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:06.542641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac50d-ab4d-4d23-9106-d6ed884aaba5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:06.542949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac50d-ab4d-4d23-9106-d6ed884aaba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:06.544511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ac50d-ab4d-4d23-9106-d6ed884aaba5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:06.544638 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7602b0d-1f76-4dd0-80ac-e1ed5c42b002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:06.545272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7602b0d-1f76-4dd0-80ac-e1ed5c42b002 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:06.545614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7602b0d-1f76-4dd0-80ac-e1ed5c42b002 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:06.546176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7602b0d-1f76-4dd0-80ac-e1ed5c42b002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:06.554181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 280/549] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:06.555482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fed26-1dd3-4d53-90de-b4ba554ce846 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:06.555774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fed26-1dd3-4d53-90de-b4ba554ce846 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:06.558287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7602b0d-1f76-4dd0-80ac-e1ed5c42b002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:06.559813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7602b0d-1f76-4dd0-80ac-e1ed5c42b002 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:06.560527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fed26-1dd3-4d53-90de-b4ba554ce846 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:06.562274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 281/550] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:06.562999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fed26-1dd3-4d53-90de-b4ba554ce846 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:06.575162 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-e7602b0d-1f76-4dd0-80ac-e1ed5c42b002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:06.585424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:06.586699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:06.587910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:06.587910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:06.587910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:06.587910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 270/551] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:06.588198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2298b8-529b-4732-be79-5df649c360a3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:06.588316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2298b8-529b-4732-be79-5df649c360a3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:06.588592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2298b8-529b-4732-be79-5df649c360a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:06.611550 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4e2298b8-529b-4732-be79-5df649c360a3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:06.612753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4e2298b8-529b-4732-be79-5df649c360a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:06.670545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:06.693148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fed26-1dd3-4d53-90de-b4ba554ce846 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:06.696688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 271/552] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:06.696688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2298b8-529b-4732-be79-5df649c360a3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:06.736277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b8bed-39bf-45b6-ae04-3f9eda834a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:06.737189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b8bed-39bf-45b6-ae04-3f9eda834a32 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:06.737791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b8bed-39bf-45b6-ae04-3f9eda834a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:06.738211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b8bed-39bf-45b6-ae04-3f9eda834a32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:06.750751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b8bed-39bf-45b6-ae04-3f9eda834a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:06.752754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b8bed-39bf-45b6-ae04-3f9eda834a32 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e07b8bed-39bf-45b6-ae04-3f9eda834a32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:06.785967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:06.787091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:06.788094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:06.788094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:06.788094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:06.788094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 282/553] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:06.793263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b8bed-39bf-45b6-ae04-3f9eda834a32 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:06.814242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2298b8-529b-4732-be79-5df649c360a3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:06.846139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:06.852410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84c50e-491e-4963-aab8-c72f803729b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:06.852772 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84c50e-491e-4963-aab8-c72f803729b6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:06.854750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84c50e-491e-4963-aab8-c72f803729b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:06.854750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84c50e-491e-4963-aab8-c72f803729b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:06.854750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84c50e-491e-4963-aab8-c72f803729b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:06.854750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84c50e-491e-4963-aab8-c72f803729b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:06.855684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84c50e-491e-4963-aab8-c72f803729b6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:06.868383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 272/554] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:06.874335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84c50e-491e-4963-aab8-c72f803729b6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:06.908134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b8bed-39bf-45b6-ae04-3f9eda834a32 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:06.945840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21103e-154b-43fb-8c35-4be0482d4a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:06.946792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21103e-154b-43fb-8c35-4be0482d4a7f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:06.947380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21103e-154b-43fb-8c35-4be0482d4a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:06.947836 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21103e-154b-43fb-8c35-4be0482d4a7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:06.950751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:06.961211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21103e-154b-43fb-8c35-4be0482d4a7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0f21103e-154b-43fb-8c35-4be0482d4a7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:06.966456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:06.967479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:06.968273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:06.972844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 283/555] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:06.974062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21103e-154b-43fb-8c35-4be0482d4a7f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:06.983717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84c50e-491e-4963-aab8-c72f803729b6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:07.021660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f53941-52b7-4d23-90aa-7315c0a3ee09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:07.022575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f53941-52b7-4d23-90aa-7315c0a3ee09 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.023156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f53941-52b7-4d23-90aa-7315c0a3ee09 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.023617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f53941-52b7-4d23-90aa-7315c0a3ee09 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.024496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f53941-52b7-4d23-90aa-7315c0a3ee09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:07.024990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f53941-52b7-4d23-90aa-7315c0a3ee09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:07.026775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f53941-52b7-4d23-90aa-7315c0a3ee09 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:07.044924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 273/556] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:07.046107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f53941-52b7-4d23-90aa-7315c0a3ee09 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:07.086868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21103e-154b-43fb-8c35-4be0482d4a7f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:07.100461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:07.124828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484f562-3574-4d9d-9cb9-ca022d6f457e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:07.125736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484f562-3574-4d9d-9cb9-ca022d6f457e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.126023 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484f562-3574-4d9d-9cb9-ca022d6f457e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.126306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484f562-3574-4d9d-9cb9-ca022d6f457e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.136865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484f562-3574-4d9d-9cb9-ca022d6f457e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:07.138251 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484f562-3574-4d9d-9cb9-ca022d6f457e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a484f562-3574-4d9d-9cb9-ca022d6f457e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:07.151307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:07.152227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 284/557] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Aug 11 09:41:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:07.153608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484f562-3574-4d9d-9cb9-ca022d6f457e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:07.160326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f53941-52b7-4d23-90aa-7315c0a3ee09 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:07.209683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b550b-4316-4b9b-9113-f223a99e0d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:07.210554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b550b-4316-4b9b-9113-f223a99e0d55 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.210975 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b550b-4316-4b9b-9113-f223a99e0d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.211519 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b550b-4316-4b9b-9113-f223a99e0d55 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.212445 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b550b-4316-4b9b-9113-f223a99e0d55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:07.212880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b550b-4316-4b9b-9113-f223a99e0d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:07.214429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b550b-4316-4b9b-9113-f223a99e0d55 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:07.219706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:07.225478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 274/558] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:07.232297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b550b-4316-4b9b-9113-f223a99e0d55 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:07.275775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484f562-3574-4d9d-9cb9-ca022d6f457e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:07.313194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de9ec3-2107-431b-8af1-c3ea50cf6f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:07.313940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de9ec3-2107-431b-8af1-c3ea50cf6f40 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.314292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de9ec3-2107-431b-8af1-c3ea50cf6f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.314639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de9ec3-2107-431b-8af1-c3ea50cf6f40 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.323319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de9ec3-2107-431b-8af1-c3ea50cf6f40 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:07.324844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de9ec3-2107-431b-8af1-c3ea50cf6f40 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-67de9ec3-2107-431b-8af1-c3ea50cf6f40 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:07.337974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:07.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:07.339902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:07.339902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:07.339902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:07.339902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:07.339902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:07.339902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:07.339902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:07.339902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:07.339902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:07.339902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:07.341910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b550b-4316-4b9b-9113-f223a99e0d55 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:07.343950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 285/559] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:07.345651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de9ec3-2107-431b-8af1-c3ea50cf6f40 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:07.357320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:07.379736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765e4e2-00fe-4797-a50f-2852ab800c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:07.380697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765e4e2-00fe-4797-a50f-2852ab800c52 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.381139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765e4e2-00fe-4797-a50f-2852ab800c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.381551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765e4e2-00fe-4797-a50f-2852ab800c52 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.382474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765e4e2-00fe-4797-a50f-2852ab800c52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:07.382947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765e4e2-00fe-4797-a50f-2852ab800c52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:07.384667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765e4e2-00fe-4797-a50f-2852ab800c52 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:07.404220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 275/560] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Aug 11 09:41:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:07.405356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765e4e2-00fe-4797-a50f-2852ab800c52 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:07.461177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de9ec3-2107-431b-8af1-c3ea50cf6f40 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:07.479065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:07.499159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a092e-b8a7-4d6c-b0a0-d99df1235f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:07.500077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a092e-b8a7-4d6c-b0a0-d99df1235f39 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.500463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a092e-b8a7-4d6c-b0a0-d99df1235f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.501159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a092e-b8a7-4d6c-b0a0-d99df1235f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.514535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a092e-b8a7-4d6c-b0a0-d99df1235f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:07.514947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765e4e2-00fe-4797-a50f-2852ab800c52 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:07.516293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65a092e-b8a7-4d6c-b0a0-d99df1235f39 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b65a092e-b8a7-4d6c-b0a0-d99df1235f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:07.532841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:07.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:07.535125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:07.535125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:07.535125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:07.541565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 286/561] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:07.542544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a092e-b8a7-4d6c-b0a0-d99df1235f39 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:07.555234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee4b15-987d-4e70-844b-9f78224afb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:07.556042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee4b15-987d-4e70-844b-9f78224afb38 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.556575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee4b15-987d-4e70-844b-9f78224afb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.556984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee4b15-987d-4e70-844b-9f78224afb38 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.557876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee4b15-987d-4e70-844b-9f78224afb38 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:07.558318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee4b15-987d-4e70-844b-9f78224afb38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:07.560065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee4b15-987d-4e70-844b-9f78224afb38 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:07.579560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 276/562] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Aug 11 09:41:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:07.580588 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee4b15-987d-4e70-844b-9f78224afb38 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:07.611125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:07.661999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a092e-b8a7-4d6c-b0a0-d99df1235f39 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:07.690975 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee4b15-987d-4e70-844b-9f78224afb38 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:07.699339 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba4a65-fd2f-4072-9a41-7e71e013e7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:07.699981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba4a65-fd2f-4072-9a41-7e71e013e7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.700264 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba4a65-fd2f-4072-9a41-7e71e013e7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.700539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba4a65-fd2f-4072-9a41-7e71e013e7b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.713415 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba4a65-fd2f-4072-9a41-7e71e013e7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:07.714875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba4a65-fd2f-4072-9a41-7e71e013e7b7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:07.727867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4359e4-1c04-4b4a-84ed-f4edaced4707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:07.728542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4359e4-1c04-4b4a-84ed-f4edaced4707 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.728819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4359e4-1c04-4b4a-84ed-f4edaced4707 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.729116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4359e4-1c04-4b4a-84ed-f4edaced4707 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.729762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4359e4-1c04-4b4a-84ed-f4edaced4707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a4ba4a65-fd2f-4072-9a41-7e71e013e7b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:07.729923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:07.730801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:07.731668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:07.731668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:07.731668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:07.731835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4359e4-1c04-4b4a-84ed-f4edaced4707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:07.731835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4359e4-1c04-4b4a-84ed-f4edaced4707 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:07.735801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 287/563] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:07.736733 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba4a65-fd2f-4072-9a41-7e71e013e7b7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:07.748553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 277/564] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:07.748553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcb95d-9b27-4846-a5bf-74be5ebb1131 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.748553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcb95d-9b27-4846-a5bf-74be5ebb1131 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.748553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcb95d-9b27-4846-a5bf-74be5ebb1131 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.748553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 278/565] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:07.748553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcb95d-9b27-4846-a5bf-74be5ebb1131 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:07.765637 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:07.845113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:07.850189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba4a65-fd2f-4072-9a41-7e71e013e7b7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:07.861842 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcb95d-9b27-4846-a5bf-74be5ebb1131 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:07.890801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efedf6-c682-4118-b43b-fba24ef67d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:07.891408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efedf6-c682-4118-b43b-fba24ef67d90 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.891689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efedf6-c682-4118-b43b-fba24ef67d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.891958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efedf6-c682-4118-b43b-fba24ef67d90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.901821 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4dc21-b3ad-491c-8e84-7465e6e4f918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:07.902427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4dc21-b3ad-491c-8e84-7465e6e4f918 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:07.902710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4dc21-b3ad-491c-8e84-7465e6e4f918 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:07.902979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4dc21-b3ad-491c-8e84-7465e6e4f918 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:07.903635 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4dc21-b3ad-491c-8e84-7465e6e4f918 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:07.903933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4dc21-b3ad-491c-8e84-7465e6e4f918 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:07.905332 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4dc21-b3ad-491c-8e84-7465e6e4f918 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:07.918095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 279/566] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:07.919355 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4dc21-b3ad-491c-8e84-7465e6e4f918 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:07.971514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efedf6-c682-4118-b43b-fba24ef67d90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:07.973924 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efedf6-c682-4118-b43b-fba24ef67d90 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:08.002469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:08.029003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4dc21-b3ad-491c-8e84-7465e6e4f918 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:08.069350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0976ab-f7cd-4609-ba4c-24b5ab21fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:08.070073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0976ab-f7cd-4609-ba4c-24b5ab21fd02 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.070425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0976ab-f7cd-4609-ba4c-24b5ab21fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.070765 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0976ab-f7cd-4609-ba4c-24b5ab21fd02 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.078190 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0976ab-f7cd-4609-ba4c-24b5ab21fd02 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:08.079693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0976ab-f7cd-4609-ba4c-24b5ab21fd02 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-5c0976ab-f7cd-4609-ba4c-24b5ab21fd02 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:08.093036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:08.094218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:08.095455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:08.095455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:08.095455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:08.095455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:08.095455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:08.095455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:08.095455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:08.095455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:08.095455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 11 09:41:08.095455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:08.105263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 280/567] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:08.105263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671ad79-9299-4709-a137-72bf4b85f541 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.105263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671ad79-9299-4709-a137-72bf4b85f541 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.105263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671ad79-9299-4709-a137-72bf4b85f541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.105779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 288/568] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:08.105779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efedf6-c682-4118-b43b-fba24ef67d90 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:08.121463 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-8671ad79-9299-4709-a137-72bf4b85f541 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:08.122703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8671ad79-9299-4709-a137-72bf4b85f541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:08.124659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:08.190735 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 281/569] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:08.191575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671ad79-9299-4709-a137-72bf4b85f541 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:08.221495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efedf6-c682-4118-b43b-fba24ef67d90 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:08.242421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:08.255925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b652601-b7b8-4cf7-9c79-0b5f0509b4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:08.256547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b652601-b7b8-4cf7-9c79-0b5f0509b4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.256824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b652601-b7b8-4cf7-9c79-0b5f0509b4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.257128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b652601-b7b8-4cf7-9c79-0b5f0509b4a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.263467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b652601-b7b8-4cf7-9c79-0b5f0509b4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:08.264888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b652601-b7b8-4cf7-9c79-0b5f0509b4a7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6b652601-b7b8-4cf7-9c79-0b5f0509b4a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:08.276544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:08.277628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:08.278528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:08.278528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:08.278528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:08.278528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:08.278528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:08.278528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:08.278528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:08.278528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:08.278528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 289/570] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:08.278998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b652601-b7b8-4cf7-9c79-0b5f0509b4a7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:08.292208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671ad79-9299-4709-a137-72bf4b85f541 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:08.326813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dce00-cfb4-405f-8efa-6d3bf7205646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:08.327432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dce00-cfb4-405f-8efa-6d3bf7205646 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.327709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dce00-cfb4-405f-8efa-6d3bf7205646 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.327982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dce00-cfb4-405f-8efa-6d3bf7205646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.334715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:08.338203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dce00-cfb4-405f-8efa-6d3bf7205646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:08.339727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dce00-cfb4-405f-8efa-6d3bf7205646 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-074dce00-cfb4-405f-8efa-6d3bf7205646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:08.354168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:08.359731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:08.360840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:08.360840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:08.360840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:08.360840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 282/571] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:08.360840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cacd3-bfa9-458f-b537-a5d30068999a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.360840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cacd3-bfa9-458f-b537-a5d30068999a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.360840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cacd3-bfa9-458f-b537-a5d30068999a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.360840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 283/572] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:08.360840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cacd3-bfa9-458f-b537-a5d30068999a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:08.385222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b652601-b7b8-4cf7-9c79-0b5f0509b4a7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:08.418857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa747301-e8d6-4fe9-83af-b143309ece87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:08.419508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa747301-e8d6-4fe9-83af-b143309ece87 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.419789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa747301-e8d6-4fe9-83af-b143309ece87 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.420065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa747301-e8d6-4fe9-83af-b143309ece87 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.420815 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa747301-e8d6-4fe9-83af-b143309ece87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:08.421151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa747301-e8d6-4fe9-83af-b143309ece87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:08.422682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa747301-e8d6-4fe9-83af-b143309ece87 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:08.439956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 290/573] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Aug 11 09:41:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:08.440758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa747301-e8d6-4fe9-83af-b143309ece87 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:08.464143 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cacd3-bfa9-458f-b537-a5d30068999a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:08.482395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:08.497807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab6485-ae1e-4fbf-a6b7-f01282ec5bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:08.498435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab6485-ae1e-4fbf-a6b7-f01282ec5bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.498712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab6485-ae1e-4fbf-a6b7-f01282ec5bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.498989 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab6485-ae1e-4fbf-a6b7-f01282ec5bd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.509040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab6485-ae1e-4fbf-a6b7-f01282ec5bd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9aab6485-ae1e-4fbf-a6b7-f01282ec5bd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:08.514179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:08.515252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:08.516172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 284/574] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:08.517004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab6485-ae1e-4fbf-a6b7-f01282ec5bd8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:08.543257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa747301-e8d6-4fe9-83af-b143309ece87 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:08.569315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:08.577366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef773614-e8af-41fd-82dc-9135b4e0b2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:08.577968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773614-e8af-41fd-82dc-9135b4e0b2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.578245 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773614-e8af-41fd-82dc-9135b4e0b2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.578515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef773614-e8af-41fd-82dc-9135b4e0b2f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.582728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef773614-e8af-41fd-82dc-9135b4e0b2f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:08.582728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef773614-e8af-41fd-82dc-9135b4e0b2f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:08.582728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef773614-e8af-41fd-82dc-9135b4e0b2f9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:08.592396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 291/575] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:08.597485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18805dd-f48a-4eb5-a70a-ad8dcc890c20 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.597485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18805dd-f48a-4eb5-a70a-ad8dcc890c20 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.597485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18805dd-f48a-4eb5-a70a-ad8dcc890c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.611059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d18805dd-f48a-4eb5-a70a-ad8dcc890c20 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:08.612199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d18805dd-f48a-4eb5-a70a-ad8dcc890c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:08.619361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab6485-ae1e-4fbf-a6b7-f01282ec5bd8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:08.653875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402af91-48db-4b5c-b433-ad51bdf2ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:08.654479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402af91-48db-4b5c-b433-ad51bdf2ef12 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.654755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402af91-48db-4b5c-b433-ad51bdf2ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.655028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402af91-48db-4b5c-b433-ad51bdf2ef12 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.655673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402af91-48db-4b5c-b433-ad51bdf2ef12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:08.655974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402af91-48db-4b5c-b433-ad51bdf2ef12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:08.657584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402af91-48db-4b5c-b433-ad51bdf2ef12 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:08.673898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 285/576] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:08.673898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402af91-48db-4b5c-b433-ad51bdf2ef12 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:08.680541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 292/577] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:08.681415 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18805dd-f48a-4eb5-a70a-ad8dcc890c20 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:08.709968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:08.772654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402af91-48db-4b5c-b433-ad51bdf2ef12 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:08.781902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18805dd-f48a-4eb5-a70a-ad8dcc890c20 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:08.782834 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:08.805371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6a6da-0575-4413-8a91-58202c2cdddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:08.806041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6a6da-0575-4413-8a91-58202c2cdddc None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.806317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6a6da-0575-4413-8a91-58202c2cdddc None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.806584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6a6da-0575-4413-8a91-58202c2cdddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.816508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6a6da-0575-4413-8a91-58202c2cdddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:08.818336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e30c5d-9a84-4c95-8a90-fec7b1eda8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:08.818806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6a6da-0575-4413-8a91-58202c2cdddc None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:08.819085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30c5d-9a84-4c95-8a90-fec7b1eda8ec None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.819365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30c5d-9a84-4c95-8a90-fec7b1eda8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.819633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30c5d-9a84-4c95-8a90-fec7b1eda8ec None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.820319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e30c5d-9a84-4c95-8a90-fec7b1eda8ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:08.820623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e30c5d-9a84-4c95-8a90-fec7b1eda8ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:08.821992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e30c5d-9a84-4c95-8a90-fec7b1eda8ec None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:08.830149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 293/578] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:08.830946 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e30c5d-9a84-4c95-8a90-fec7b1eda8ec None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-8cd6a6da-0575-4413-8a91-58202c2cdddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:08.833809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:08.834757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:08.835642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:08.835642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:08.835642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:08.837322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 286/579] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:08.838185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6a6da-0575-4413-8a91-58202c2cdddc None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:08.931255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:08.940321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e30c5d-9a84-4c95-8a90-fec7b1eda8ec None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:08.948833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6a6da-0575-4413-8a91-58202c2cdddc None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:08.975708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5374d33-88fb-4427-b8d6-075e87346796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:08.976287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da09f42-8b69-41cf-9b0b-d18c224aceae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:08.976529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5374d33-88fb-4427-b8d6-075e87346796 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.976743 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da09f42-8b69-41cf-9b0b-d18c224aceae None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:08.977029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5374d33-88fb-4427-b8d6-075e87346796 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.977126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da09f42-8b69-41cf-9b0b-d18c224aceae None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:08.977311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da09f42-8b69-41cf-9b0b-d18c224aceae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.977400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5374d33-88fb-4427-b8d6-075e87346796 None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/34fa13a87ebe4b789ca1884e5aeb9237/roles/c6c1ea7ecf3d4e00a138f97d91b14f73` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:08.984913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da09f42-8b69-41cf-9b0b-d18c224aceae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:08.986385 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da09f42-8b69-41cf-9b0b-d18c224aceae None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:08.992902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5374d33-88fb-4427-b8d6-075e87346796 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=34fa13a87ebe4b789ca1884e5aeb9237, role_id=c6c1ea7ecf3d4e00a138f97d91b14f73)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:08.994368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5374d33-88fb-4427-b8d6-075e87346796 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-4da09f42-8b69-41cf-9b0b-d18c224aceae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:08.999632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:09.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:09.001796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:09.001796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:09.001796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:09.001796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:09.001796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:09.001796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:09.001796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:09.001796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:09.001796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 287/580] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:09.001796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da09f42-8b69-41cf-9b0b-d18c224aceae None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:09.031674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:09.075234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 294/581] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:41:08 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/34fa13a87ebe4b789ca1884e5aeb9237/roles/c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:41:09.076123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5374d33-88fb-4427-b8d6-075e87346796 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:09.125139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da09f42-8b69-41cf-9b0b-d18c224aceae None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:09.167437 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01776a-8ac8-4dcd-ae33-14350ce0e409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:09.168107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01776a-8ac8-4dcd-ae33-14350ce0e409 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:09.168383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01776a-8ac8-4dcd-ae33-14350ce0e409 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:09.168655 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01776a-8ac8-4dcd-ae33-14350ce0e409 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:09.184532 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01776a-8ac8-4dcd-ae33-14350ce0e409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-ea01776a-8ac8-4dcd-ae33-14350ce0e409 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:09.187075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:09.188016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:09.188841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 288/582] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:09.189547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01776a-8ac8-4dcd-ae33-14350ce0e409 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:09.215285 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5374d33-88fb-4427-b8d6-075e87346796 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:09.258850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c7d4e-d30a-43e2-b35b-db4f786bdaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:09.259489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c7d4e-d30a-43e2-b35b-db4f786bdaad None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:09.259766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c7d4e-d30a-43e2-b35b-db4f786bdaad None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:09.260034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84c7d4e-d30a-43e2-b35b-db4f786bdaad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:09.271935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:09.275230 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c7d4e-d30a-43e2-b35b-db4f786bdaad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:09.277572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84c7d4e-d30a-43e2-b35b-db4f786bdaad None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e84c7d4e-d30a-43e2-b35b-db4f786bdaad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:09.296645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:09.297784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:09.298640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:09.298640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:09.298640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:09.305859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 295/583] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:09.307631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c7d4e-d30a-43e2-b35b-db4f786bdaad None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:09.307903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:09.331331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01776a-8ac8-4dcd-ae33-14350ce0e409 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:09.374287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e6e83-ee80-4a56-b6c5-143f07d1bb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:09.375161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e6e83-ee80-4a56-b6c5-143f07d1bb66 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:09.375500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e6e83-ee80-4a56-b6c5-143f07d1bb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:09.375832 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e6e83-ee80-4a56-b6c5-143f07d1bb66 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:09.376429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e6e83-ee80-4a56-b6c5-143f07d1bb66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:09.376730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e6e83-ee80-4a56-b6c5-143f07d1bb66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:09.378172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e6e83-ee80-4a56-b6c5-143f07d1bb66 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:09.392392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 289/584] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:09.393878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e6e83-ee80-4a56-b6c5-143f07d1bb66 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:09.435768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84c7d4e-d30a-43e2-b35b-db4f786bdaad None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:09.475192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d018-4ac8-47dd-840b-c81409ea2df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:09.475805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d018-4ac8-47dd-840b-c81409ea2df2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:09.476089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d018-4ac8-47dd-840b-c81409ea2df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:09.476360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d018-4ac8-47dd-840b-c81409ea2df2 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:09.477082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d018-4ac8-47dd-840b-c81409ea2df2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:09.477390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d018-4ac8-47dd-840b-c81409ea2df2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:09.478809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d018-4ac8-47dd-840b-c81409ea2df2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:09.498518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 296/585] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:09.498518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d018-4ac8-47dd-840b-c81409ea2df2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:09.551051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e6e83-ee80-4a56-b6c5-143f07d1bb66 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:09.564936 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:09.590121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370a91a-a266-442b-8afc-7ce70c746a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:09.590738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a91a-a266-442b-8afc-7ce70c746a60 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:09.591016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a91a-a266-442b-8afc-7ce70c746a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:09.591288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a91a-a266-442b-8afc-7ce70c746a60 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:09.591955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370a91a-a266-442b-8afc-7ce70c746a60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:09.592260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370a91a-a266-442b-8afc-7ce70c746a60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:09.593783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370a91a-a266-442b-8afc-7ce70c746a60 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:09.605460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 290/586] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:09.606416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370a91a-a266-442b-8afc-7ce70c746a60 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:09.611673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:09.632904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d018-4ac8-47dd-840b-c81409ea2df2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:09.696641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aec670-21da-4305-be73-5e6ff8dcf2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:09.697536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aec670-21da-4305-be73-5e6ff8dcf2ab None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:09.697998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aec670-21da-4305-be73-5e6ff8dcf2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:09.698467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aec670-21da-4305-be73-5e6ff8dcf2ab None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:09.699394 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aec670-21da-4305-be73-5e6ff8dcf2ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:09.699898 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aec670-21da-4305-be73-5e6ff8dcf2ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:09.702365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aec670-21da-4305-be73-5e6ff8dcf2ab None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:09.727696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 297/587] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:09.727696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aec670-21da-4305-be73-5e6ff8dcf2ab None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:09.762743 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370a91a-a266-442b-8afc-7ce70c746a60 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:09.811773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e263b-d0e9-4954-a79c-38ee95b1d59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:09.814799 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e263b-d0e9-4954-a79c-38ee95b1d59a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:09.815154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e263b-d0e9-4954-a79c-38ee95b1d59a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:09.815495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e263b-d0e9-4954-a79c-38ee95b1d59a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:09.825978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e263b-d0e9-4954-a79c-38ee95b1d59a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:09.827724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e263b-d0e9-4954-a79c-38ee95b1d59a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-467e263b-d0e9-4954-a79c-38ee95b1d59a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:09.840861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:09.842000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:09.843061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:09.843061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:09.843061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:09.843061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:09.843061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:09.843061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:09.843061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:09.843061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:09.848062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 291/588] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:09.848062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e263b-d0e9-4954-a79c-38ee95b1d59a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:09.858084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aec670-21da-4305-be73-5e6ff8dcf2ab None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:09.873165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:09.894981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45875e-d2cd-40a6-a701-1f30b9bbb490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:09.895844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45875e-d2cd-40a6-a701-1f30b9bbb490 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:09.896126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45875e-d2cd-40a6-a701-1f30b9bbb490 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:09.896402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45875e-d2cd-40a6-a701-1f30b9bbb490 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:09.897176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45875e-d2cd-40a6-a701-1f30b9bbb490 None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=915ee44218064af896020d008ff31957) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:09.897485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45875e-d2cd-40a6-a701-1f30b9bbb490 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:09.898911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45875e-d2cd-40a6-a701-1f30b9bbb490 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:09.905089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 298/589] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:09 2022] GET /identity/v3/role_assignments?user.id=915ee44218064af896020d008ff31957&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:09.905884 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45875e-d2cd-40a6-a701-1f30b9bbb490 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:09.922880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:09.976469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e263b-d0e9-4954-a79c-38ee95b1d59a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:10.015399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1ea46-20ae-47d7-bb73-ba3fb4af6b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:10.016040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ea46-20ae-47d7-bb73-ba3fb4af6b72 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:10.016320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ea46-20ae-47d7-bb73-ba3fb4af6b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:10.016591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1ea46-20ae-47d7-bb73-ba3fb4af6b72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:10.029874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1ea46-20ae-47d7-bb73-ba3fb4af6b72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:10.031589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1ea46-20ae-47d7-bb73-ba3fb4af6b72 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-93a1ea46-20ae-47d7-bb73-ba3fb4af6b72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:10.050771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:10.052402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:10.053291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:10.053291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:10.053291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:10.057554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 292/590] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:10.058529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1ea46-20ae-47d7-bb73-ba3fb4af6b72 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:10.064163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45875e-d2cd-40a6-a701-1f30b9bbb490 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:10.108151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085cedb-b534-40d7-bde2-1a68460dc011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:10.109189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085cedb-b534-40d7-bde2-1a68460dc011 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:10.109637 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085cedb-b534-40d7-bde2-1a68460dc011 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:10.110052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085cedb-b534-40d7-bde2-1a68460dc011 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:10.120562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085cedb-b534-40d7-bde2-1a68460dc011 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:10.122982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085cedb-b534-40d7-bde2-1a68460dc011 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-8085cedb-b534-40d7-bde2-1a68460dc011 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:10.138158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:10.139111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:10.140021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:10.140021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:10.140021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:10.140021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:10.140021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:10.140021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:10.140021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:10.140021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:10.140021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 09:41:10.140021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:10.141305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 299/591] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:10.142307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085cedb-b534-40d7-bde2-1a68460dc011 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:10.172154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1ea46-20ae-47d7-bb73-ba3fb4af6b72 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:10.187658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:10.197066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:10.216400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8d585-0074-43f7-a22c-5409be3a0cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:10.217332 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8d585-0074-43f7-a22c-5409be3a0cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:10.217778 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8d585-0074-43f7-a22c-5409be3a0cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:10.218218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8d585-0074-43f7-a22c-5409be3a0cf4 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:10.219093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8d585-0074-43f7-a22c-5409be3a0cf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:10.219601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8d585-0074-43f7-a22c-5409be3a0cf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:10.221339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8d585-0074-43f7-a22c-5409be3a0cf4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:10.237548 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 293/592] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:10.238683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8d585-0074-43f7-a22c-5409be3a0cf4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:10.274216 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085cedb-b534-40d7-bde2-1a68460dc011 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:10.316967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3787c0e-0b07-4898-b964-fafd2b5ea034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:10.318150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3787c0e-0b07-4898-b964-fafd2b5ea034 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:10.318526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3787c0e-0b07-4898-b964-fafd2b5ea034 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:10.318912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3787c0e-0b07-4898-b964-fafd2b5ea034 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:10.320476 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3787c0e-0b07-4898-b964-fafd2b5ea034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:10.320892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3787c0e-0b07-4898-b964-fafd2b5ea034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:10.322396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3787c0e-0b07-4898-b964-fafd2b5ea034 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:10.341050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 300/593] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:10.341050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3787c0e-0b07-4898-b964-fafd2b5ea034 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:10.358000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8d585-0074-43f7-a22c-5409be3a0cf4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:10.400059 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21c052-82b0-4b3b-b258-71014e9f7635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:10.400914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21c052-82b0-4b3b-b258-71014e9f7635 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:10.401411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21c052-82b0-4b3b-b258-71014e9f7635 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:10.401828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21c052-82b0-4b3b-b258-71014e9f7635 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:10.402875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21c052-82b0-4b3b-b258-71014e9f7635 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:10.403339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21c052-82b0-4b3b-b258-71014e9f7635 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:10.405048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21c052-82b0-4b3b-b258-71014e9f7635 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:10.450844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 294/594] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Aug 11 09:41:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:10.451939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21c052-82b0-4b3b-b258-71014e9f7635 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:10.473707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:10.480336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3787c0e-0b07-4898-b964-fafd2b5ea034 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:10.489757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:10.517409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec1d7d-666d-46b9-a267-1f644c8a1fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:10.518071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec1d7d-666d-46b9-a267-1f644c8a1fbe None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:10.518405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec1d7d-666d-46b9-a267-1f644c8a1fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:10.518616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec1d7d-666d-46b9-a267-1f644c8a1fbe None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/d83448e4c6704f7c8771132bd67eaa1d/roles/c6c1ea7ecf3d4e00a138f97d91b14f73` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:10.535942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec1d7d-666d-46b9-a267-1f644c8a1fbe None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=d83448e4c6704f7c8771132bd67eaa1d, role_id=c6c1ea7ecf3d4e00a138f97d91b14f73)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:10.537226 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec1d7d-666d-46b9-a267-1f644c8a1fbe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:10.561609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21c052-82b0-4b3b-b258-71014e9f7635 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:10.599437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 301/595] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:41:10 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/d83448e4c6704f7c8771132bd67eaa1d/roles/c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:41:10.600986 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec1d7d-666d-46b9-a267-1f644c8a1fbe None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:10.603921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52e94-29de-4555-8cda-e615ecad4ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:10.604643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52e94-29de-4555-8cda-e615ecad4ecf None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:10.605084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52e94-29de-4555-8cda-e615ecad4ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:10.605574 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52e94-29de-4555-8cda-e615ecad4ecf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:10.622249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c52e94-29de-4555-8cda-e615ecad4ecf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:10.623774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c52e94-29de-4555-8cda-e615ecad4ecf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-43c52e94-29de-4555-8cda-e615ecad4ecf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:10.642687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:10.643754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:10.644782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:10.644782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:10.644782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:10.644782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:10.644782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:10.644782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:10.644782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:10.644782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:10.644782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 09:41:10.644782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:10.644782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 295/596] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:10.655370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52e94-29de-4555-8cda-e615ecad4ecf None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:10.741852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec1d7d-666d-46b9-a267-1f644c8a1fbe None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:10.787024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d757a4d-8ede-44cc-9b1d-287b576d8e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:10.788255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d757a4d-8ede-44cc-9b1d-287b576d8e63 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:10.788756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d757a4d-8ede-44cc-9b1d-287b576d8e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:10.789254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d757a4d-8ede-44cc-9b1d-287b576d8e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:10.802734 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d757a4d-8ede-44cc-9b1d-287b576d8e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:10.804834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d757a4d-8ede-44cc-9b1d-287b576d8e63 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:10.814297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5d757a4d-8ede-44cc-9b1d-287b576d8e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:10.825436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:10.831909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:10.832845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:10.832845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:10.832845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:10.832845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 302/597] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:10.835756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d757a4d-8ede-44cc-9b1d-287b576d8e63 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:10.861403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:10.879304 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52e94-29de-4555-8cda-e615ecad4ecf None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:10.929062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127e54d-c2bf-4ba2-84db-b1def8e937a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:10.929715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127e54d-c2bf-4ba2-84db-b1def8e937a8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:10.929995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127e54d-c2bf-4ba2-84db-b1def8e937a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:10.930270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127e54d-c2bf-4ba2-84db-b1def8e937a8 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:10.931059 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127e54d-c2bf-4ba2-84db-b1def8e937a8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:10.931313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127e54d-c2bf-4ba2-84db-b1def8e937a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:10.932744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6127e54d-c2bf-4ba2-84db-b1def8e937a8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:10.954789 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 296/598] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Aug 11 09:41:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:10.965650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127e54d-c2bf-4ba2-84db-b1def8e937a8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:10.971354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d757a4d-8ede-44cc-9b1d-287b576d8e63 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:11.020769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fdc81-5748-4a80-b7db-5308156ef8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:11.021767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fdc81-5748-4a80-b7db-5308156ef8fe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:11.022558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fdc81-5748-4a80-b7db-5308156ef8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:11.022981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fdc81-5748-4a80-b7db-5308156ef8fe None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:11.024112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fdc81-5748-4a80-b7db-5308156ef8fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:11.024564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fdc81-5748-4a80-b7db-5308156ef8fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:11.026379 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fdc81-5748-4a80-b7db-5308156ef8fe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:11.043654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 303/599] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:11.048868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fdc81-5748-4a80-b7db-5308156ef8fe None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:11.118972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:11.167906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127e54d-c2bf-4ba2-84db-b1def8e937a8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:11.168475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fdc81-5748-4a80-b7db-5308156ef8fe None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:11.215533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c0c32-1049-4735-8ad9-acd8081f3cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:11.216075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03507991-80af-4526-9bf2-473c955dd462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:11.216206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c0c32-1049-4735-8ad9-acd8081f3cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:11.216431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c0c32-1049-4735-8ad9-acd8081f3cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:11.216703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c0c32-1049-4735-8ad9-acd8081f3cc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:11.217331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03507991-80af-4526-9bf2-473c955dd462 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:11.217815 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03507991-80af-4526-9bf2-473c955dd462 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:11.218227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03507991-80af-4526-9bf2-473c955dd462 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:11.225994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03507991-80af-4526-9bf2-473c955dd462 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:11.227573 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c0c32-1049-4735-8ad9-acd8081f3cc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:11.227739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03507991-80af-4526-9bf2-473c955dd462 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:11.229044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c0c32-1049-4735-8ad9-acd8081f3cc9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-03507991-80af-4526-9bf2-473c955dd462 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:11.240351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:11.241437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:11.242344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:11.242344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:11.242344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:11.242344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:11.242344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:11.242344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:11.242344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:11.242344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-2d3c0c32-1049-4735-8ad9-acd8081f3cc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:11.244382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:11.245267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:11.246131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:11.246131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:11.246131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:11.246948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 304/601] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:11.248332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87575b-5194-453c-a489-1785bab3f9cf None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:11.249092 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87575b-5194-453c-a489-1785bab3f9cf None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:11.249543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87575b-5194-453c-a489-1785bab3f9cf None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:11.250149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 297/601] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:11.250987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c0c32-1049-4735-8ad9-acd8081f3cc9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:11.251651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 305/602] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:11.252530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87575b-5194-453c-a489-1785bab3f9cf None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:11.257524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:11.357525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c0c32-1049-4735-8ad9-acd8081f3cc9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:11.362101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87575b-5194-453c-a489-1785bab3f9cf None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:11.381150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:11.393042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d4cf4-0788-4f33-b1f5-c896b78ebcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:11.393655 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d4cf4-0788-4f33-b1f5-c896b78ebcfd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:11.393950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d4cf4-0788-4f33-b1f5-c896b78ebcfd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:11.394207 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d4cf4-0788-4f33-b1f5-c896b78ebcfd None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:11.394904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d4cf4-0788-4f33-b1f5-c896b78ebcfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:11.395757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d4cf4-0788-4f33-b1f5-c896b78ebcfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:11.397227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d4cf4-0788-4f33-b1f5-c896b78ebcfd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:11.403445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50f5f7-9c12-4224-a3ed-8e38c9156e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:11.404405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50f5f7-9c12-4224-a3ed-8e38c9156e38 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:11.404933 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50f5f7-9c12-4224-a3ed-8e38c9156e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:11.405489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50f5f7-9c12-4224-a3ed-8e38c9156e38 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:11.406316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50f5f7-9c12-4224-a3ed-8e38c9156e38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:11.406822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50f5f7-9c12-4224-a3ed-8e38c9156e38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:11.408528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50f5f7-9c12-4224-a3ed-8e38c9156e38 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:11.410408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 298/603] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:11.411939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14655837-4bd8-4629-b06b-2b9e5993d4e1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:11.412145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14655837-4bd8-4629-b06b-2b9e5993d4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:11.412423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14655837-4bd8-4629-b06b-2b9e5993d4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:11.423994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 306/604] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:11.425129 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50f5f7-9c12-4224-a3ed-8e38c9156e38 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:11.435000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-14655837-4bd8-4629-b06b-2b9e5993d4e1 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:11.436234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-14655837-4bd8-4629-b06b-2b9e5993d4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:11.502996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:11.513122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 299/605] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:11.514012 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14655837-4bd8-4629-b06b-2b9e5993d4e1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:11.537593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50f5f7-9c12-4224-a3ed-8e38c9156e38 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:11.575171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ec7ca-ca4f-47e4-b9ec-bbe841e1a69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:11.575984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ec7ca-ca4f-47e4-b9ec-bbe841e1a69c None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:11.576357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ec7ca-ca4f-47e4-b9ec-bbe841e1a69c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:11.576636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ec7ca-ca4f-47e4-b9ec-bbe841e1a69c None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/029bb992f9974eec83c0fe7f9fa58826/roles/c6c1ea7ecf3d4e00a138f97d91b14f73` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:11.593036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ec7ca-ca4f-47e4-b9ec-bbe841e1a69c None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=029bb992f9974eec83c0fe7f9fa58826, role_id=c6c1ea7ecf3d4e00a138f97d91b14f73)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:11.594454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ec7ca-ca4f-47e4-b9ec-bbe841e1a69c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:11.631904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14655837-4bd8-4629-b06b-2b9e5993d4e1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:11.666840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c7230-f22b-4bc5-a6a4-ddd2c5d59187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:11.667467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c7230-f22b-4bc5-a6a4-ddd2c5d59187 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:11.667762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c7230-f22b-4bc5-a6a4-ddd2c5d59187 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:11.668033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c7230-f22b-4bc5-a6a4-ddd2c5d59187 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:11.674902 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c7230-f22b-4bc5-a6a4-ddd2c5d59187 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:11.676877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c7230-f22b-4bc5-a6a4-ddd2c5d59187 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-fa3c7230-f22b-4bc5-a6a4-ddd2c5d59187 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:11.688333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:11.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:11.690223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:11.690223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:11.690223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:11.690223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:11.690223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:11.690223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:11.690223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:11.690223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:11.690223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 300/606] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:11.690688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c7230-f22b-4bc5-a6a4-ddd2c5d59187 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:11.713099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 307/607] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:41:11 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/029bb992f9974eec83c0fe7f9fa58826/roles/c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:41:11.714111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ec7ca-ca4f-47e4-b9ec-bbe841e1a69c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:11.776330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:11.800569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:11.870961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c7230-f22b-4bc5-a6a4-ddd2c5d59187 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:11.920300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f85606-1519-4794-a97b-87907426618c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:11.923304 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f85606-1519-4794-a97b-87907426618c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:11.923759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f85606-1519-4794-a97b-87907426618c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:11.924185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f85606-1519-4794-a97b-87907426618c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:11.946217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ec7ca-ca4f-47e4-b9ec-bbe841e1a69c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:11.946684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f85606-1519-4794-a97b-87907426618c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-80f85606-1519-4794-a97b-87907426618c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:11.954210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:11.955361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:11.956461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:11.960817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 301/608] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:11.966799 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f85606-1519-4794-a97b-87907426618c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:11.996744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b1cb1-dbb9-45ec-9611-5d1683ab92bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:11.997639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b1cb1-dbb9-45ec-9611-5d1683ab92bc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:11.998023 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b1cb1-dbb9-45ec-9611-5d1683ab92bc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:11.998402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b1cb1-dbb9-45ec-9611-5d1683ab92bc None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:11.999208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b1cb1-dbb9-45ec-9611-5d1683ab92bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:11.999628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b1cb1-dbb9-45ec-9611-5d1683ab92bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:12.001401 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b1cb1-dbb9-45ec-9611-5d1683ab92bc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:12.019304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 308/609] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:12.027548 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b1cb1-dbb9-45ec-9611-5d1683ab92bc None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:12.100587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f85606-1519-4794-a97b-87907426618c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:12.115681 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:12.137659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5457f-f9e3-459f-8b80-1321d591b68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:12.138268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5457f-f9e3-459f-8b80-1321d591b68e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:12.138602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5457f-f9e3-459f-8b80-1321d591b68e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:12.138921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5457f-f9e3-459f-8b80-1321d591b68e None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:12.139606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5457f-f9e3-459f-8b80-1321d591b68e None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:12.139891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5457f-f9e3-459f-8b80-1321d591b68e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:12.141281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5457f-f9e3-459f-8b80-1321d591b68e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:12.152091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 302/610] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:12.153697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8b25d-a8dc-4698-b02e-c6d4ef31d33f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:12.154115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8b25d-a8dc-4698-b02e-c6d4ef31d33f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:12.154431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8b25d-a8dc-4698-b02e-c6d4ef31d33f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:12.155789 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 303/611] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:12.158678 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:12.160494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8b25d-a8dc-4698-b02e-c6d4ef31d33f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:12.175816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b1cb1-dbb9-45ec-9611-5d1683ab92bc None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:12.223761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c4573-6779-45df-8b11-b8f93b5d9a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:12.224912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c4573-6779-45df-8b11-b8f93b5d9a6a None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:12.225792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c4573-6779-45df-8b11-b8f93b5d9a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:12.226224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c4573-6779-45df-8b11-b8f93b5d9a6a None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/4fc6c3d1ce8645808c22d1b59b15a100/roles/c6c1ea7ecf3d4e00a138f97d91b14f73` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:12.246622 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c4573-6779-45df-8b11-b8f93b5d9a6a None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=4fc6c3d1ce8645808c22d1b59b15a100, role_id=c6c1ea7ecf3d4e00a138f97d91b14f73)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:12.249217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c4573-6779-45df-8b11-b8f93b5d9a6a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:12.279672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8b25d-a8dc-4698-b02e-c6d4ef31d33f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:12.317051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 309/612] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:41:12 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/4fc6c3d1ce8645808c22d1b59b15a100/roles/c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 09:41:12.317051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e27ba-95e2-4e8a-90ad-28c941a0e32e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:12.317051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e27ba-95e2-4e8a-90ad-28c941a0e32e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:12.317051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e27ba-95e2-4e8a-90ad-28c941a0e32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:12.333515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564bbbe-a986-4c8a-91e0-9c2ca620d8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:12.334156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564bbbe-a986-4c8a-91e0-9c2ca620d8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:12.334427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564bbbe-a986-4c8a-91e0-9c2ca620d8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:12.334695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564bbbe-a986-4c8a-91e0-9c2ca620d8b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:12.353254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-693e27ba-95e2-4e8a-90ad-28c941a0e32e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:12.356093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-693e27ba-95e2-4e8a-90ad-28c941a0e32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:12.361282 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564bbbe-a986-4c8a-91e0-9c2ca620d8b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:12.362790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564bbbe-a986-4c8a-91e0-9c2ca620d8b0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-8564bbbe-a986-4c8a-91e0-9c2ca620d8b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:12.389296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:12.390278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:12.391141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:12.391141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:12.391141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:12.391740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 304/613] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:12.397003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564bbbe-a986-4c8a-91e0-9c2ca620d8b0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:12.459864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:12.501192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 310/614] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:12.527427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e27ba-95e2-4e8a-90ad-28c941a0e32e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:12.570469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564bbbe-a986-4c8a-91e0-9c2ca620d8b0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:12.616855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:12.623412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda41239-64a1-408e-920a-4ce18640060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:12.627950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda41239-64a1-408e-920a-4ce18640060b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:12.628397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda41239-64a1-408e-920a-4ce18640060b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:12.628907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda41239-64a1-408e-920a-4ce18640060b None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:12.630260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda41239-64a1-408e-920a-4ce18640060b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:12.630654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda41239-64a1-408e-920a-4ce18640060b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:12.632117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda41239-64a1-408e-920a-4ce18640060b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:12.653220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 305/615] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:12.657377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda41239-64a1-408e-920a-4ce18640060b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:12.681948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e27ba-95e2-4e8a-90ad-28c941a0e32e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:12.742762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42092363-2b86-47e1-8cb1-7543bed25455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:12.743329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092363-2b86-47e1-8cb1-7543bed25455 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:12.743617 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092363-2b86-47e1-8cb1-7543bed25455 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:12.743892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092363-2b86-47e1-8cb1-7543bed25455 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:12.765900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42092363-2b86-47e1-8cb1-7543bed25455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-42092363-2b86-47e1-8cb1-7543bed25455 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:12.778280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:12.779450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:12.780276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 311/616] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:12.785207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42092363-2b86-47e1-8cb1-7543bed25455 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:12.833777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:12.863803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda41239-64a1-408e-920a-4ce18640060b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:12.932347 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42092363-2b86-47e1-8cb1-7543bed25455 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:12.939493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153fe72-4907-41b4-8f66-831cbf4c0866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:12.940135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153fe72-4907-41b4-8f66-831cbf4c0866 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:12.940413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153fe72-4907-41b4-8f66-831cbf4c0866 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:12.940682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153fe72-4907-41b4-8f66-831cbf4c0866 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:12.953647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153fe72-4907-41b4-8f66-831cbf4c0866 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:12.955101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153fe72-4907-41b4-8f66-831cbf4c0866 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-7153fe72-4907-41b4-8f66-831cbf4c0866 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:12.968228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:12.969478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:12.970411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:12.970411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:12.970411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:12.970411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:12.970411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:12.970411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:12.970411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:12.970411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:12.970411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:12.970411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:12.970411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 306/617] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Aug 11 09:41:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:12.981893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:12.982760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153fe72-4907-41b4-8f66-831cbf4c0866 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:12.991057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42957ebe-2502-4f58-84ca-b2e2fb110be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:12.991985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42957ebe-2502-4f58-84ca-b2e2fb110be7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:12.992556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42957ebe-2502-4f58-84ca-b2e2fb110be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:12.993123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42957ebe-2502-4f58-84ca-b2e2fb110be7 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:12.994150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42957ebe-2502-4f58-84ca-b2e2fb110be7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:12.994699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42957ebe-2502-4f58-84ca-b2e2fb110be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:12.996310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42957ebe-2502-4f58-84ca-b2e2fb110be7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:13.006424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 312/618] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:13.014103 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42957ebe-2502-4f58-84ca-b2e2fb110be7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:13.105639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153fe72-4907-41b4-8f66-831cbf4c0866 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:13.152178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42957ebe-2502-4f58-84ca-b2e2fb110be7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:13.152376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711a48b-298c-479c-8f3d-47b82bf42766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:13.152845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711a48b-298c-479c-8f3d-47b82bf42766 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:13.153152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711a48b-298c-479c-8f3d-47b82bf42766 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:13.153424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711a48b-298c-479c-8f3d-47b82bf42766 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:13.154211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711a48b-298c-479c-8f3d-47b82bf42766 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:13.154514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711a48b-298c-479c-8f3d-47b82bf42766 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:13.157140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711a48b-298c-479c-8f3d-47b82bf42766 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:13.187542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 307/619] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Aug 11 09:41:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:13.187542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b63c8-6409-4b92-85a4-add9243b076a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:13.187542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b63c8-6409-4b92-85a4-add9243b076a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:13.187542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b63c8-6409-4b92-85a4-add9243b076a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:13.189216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 308/620] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:13.189911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b63c8-6409-4b92-85a4-add9243b076a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:13.201063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1146ec-55e8-4f25-aba9-8d5d2df32467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:13.202440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1146ec-55e8-4f25-aba9-8d5d2df32467 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:13.202969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1146ec-55e8-4f25-aba9-8d5d2df32467 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:13.203505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1146ec-55e8-4f25-aba9-8d5d2df32467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:13.204396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:13.215823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1146ec-55e8-4f25-aba9-8d5d2df32467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:13.217621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1146ec-55e8-4f25-aba9-8d5d2df32467 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-fa1146ec-55e8-4f25-aba9-8d5d2df32467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:13.233659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:13.234984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:13.236062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:13.236062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:13.236062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:13.238072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 313/621] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:13.240307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958371e-58b2-4f65-9a69-fbece68b80f3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:13.240750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958371e-58b2-4f65-9a69-fbece68b80f3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:13.241294 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958371e-58b2-4f65-9a69-fbece68b80f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:13.260839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c958371e-58b2-4f65-9a69-fbece68b80f3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:13.262382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c958371e-58b2-4f65-9a69-fbece68b80f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:13.303021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:13.328917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b63c8-6409-4b92-85a4-add9243b076a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:13.337216 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 314/622] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:13.338342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958371e-58b2-4f65-9a69-fbece68b80f3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:13.382746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b925771-cbcf-481e-a5e5-76b6b744a993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:13.383696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b925771-cbcf-481e-a5e5-76b6b744a993 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:13.384272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b925771-cbcf-481e-a5e5-76b6b744a993 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:13.384762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b925771-cbcf-481e-a5e5-76b6b744a993 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:13.459336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b925771-cbcf-481e-a5e5-76b6b744a993 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:13.461185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b925771-cbcf-481e-a5e5-76b6b744a993 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:13.470031 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:13.482711 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958371e-58b2-4f65-9a69-fbece68b80f3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:13.529759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b5bce-87c9-4c60-b4dc-bafe5c982389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:13.530372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b5bce-87c9-4c60-b4dc-bafe5c982389 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:13.530650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b5bce-87c9-4c60-b4dc-bafe5c982389 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:13.530921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b5bce-87c9-4c60-b4dc-bafe5c982389 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:13.531818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b5bce-87c9-4c60-b4dc-bafe5c982389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:13.532121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b5bce-87c9-4c60-b4dc-bafe5c982389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:13.534074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b5bce-87c9-4c60-b4dc-bafe5c982389 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:13.554822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 315/623] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:13.554822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b5bce-87c9-4c60-b4dc-bafe5c982389 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:13.575689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 309/624] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:13.578329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b925771-cbcf-481e-a5e5-76b6b744a993 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:13.606780 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:13.694946 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b925771-cbcf-481e-a5e5-76b6b744a993 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:13.697350 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b5bce-87c9-4c60-b4dc-bafe5c982389 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:13.703206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:13.737175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e595f-33d6-4554-b401-b38f77bfcf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:13.738008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e595f-33d6-4554-b401-b38f77bfcf20 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:13.738384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e595f-33d6-4554-b401-b38f77bfcf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:13.738607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac6f42-75de-401a-a72f-d27135e181a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:13.740202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6f42-75de-401a-a72f-d27135e181a8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:13.740622 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6f42-75de-401a-a72f-d27135e181a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:13.741057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6f42-75de-401a-a72f-d27135e181a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:13.741217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e595f-33d6-4554-b401-b38f77bfcf20 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:13.749458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e595f-33d6-4554-b401-b38f77bfcf20 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:13.751198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e595f-33d6-4554-b401-b38f77bfcf20 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:13.754479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac6f42-75de-401a-a72f-d27135e181a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-dcac6f42-75de-401a-a72f-d27135e181a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:13.759462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:13.760392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:13.761239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-282e595f-33d6-4554-b401-b38f77bfcf20 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:13.763535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:13.764418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:13.765659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:13.765659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:13.765659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:13.765659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:13.765659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:13.765659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:13.765659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:13.765659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:13.765659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:13.765659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:13.766600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 316/625] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:13.767546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac6f42-75de-401a-a72f-d27135e181a8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:13.774238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 310/626] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:13.774238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e595f-33d6-4554-b401-b38f77bfcf20 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:13.892804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:13.898649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e595f-33d6-4554-b401-b38f77bfcf20 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:13.904511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac6f42-75de-401a-a72f-d27135e181a8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:13.939027 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a9276-35b3-4d67-8822-a15c33a6fca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:13.940076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a9276-35b3-4d67-8822-a15c33a6fca5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:13.940690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a9276-35b3-4d67-8822-a15c33a6fca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:13.941195 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a9276-35b3-4d67-8822-a15c33a6fca5 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:13.942603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a9276-35b3-4d67-8822-a15c33a6fca5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:13.943183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a9276-35b3-4d67-8822-a15c33a6fca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:13.945202 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a9276-35b3-4d67-8822-a15c33a6fca5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:13.946066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a4adc-ce86-473c-a16e-24e79950ee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:13.946768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a4adc-ce86-473c-a16e-24e79950ee37 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:13.947115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a4adc-ce86-473c-a16e-24e79950ee37 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:13.947634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a4adc-ce86-473c-a16e-24e79950ee37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:13.955640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 311/627] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:13.956909 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a9276-35b3-4d67-8822-a15c33a6fca5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:13.965524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a4adc-ce86-473c-a16e-24e79950ee37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:13.966965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a4adc-ce86-473c-a16e-24e79950ee37 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-654a4adc-ce86-473c-a16e-24e79950ee37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:13.986411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:13.987491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:13.988369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:13.988369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:13.988369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:13.992190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 317/628] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:13.993358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a4adc-ce86-473c-a16e-24e79950ee37 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:13.995831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:14.089666 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a9276-35b3-4d67-8822-a15c33a6fca5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:14.113151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a4adc-ce86-473c-a16e-24e79950ee37 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:14.124855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5947a0-f267-4d55-abc5-73f30f61715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:14.125714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5947a0-f267-4d55-abc5-73f30f61715a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.126061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5947a0-f267-4d55-abc5-73f30f61715a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.126433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5947a0-f267-4d55-abc5-73f30f61715a None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.127363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5947a0-f267-4d55-abc5-73f30f61715a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:14.127753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5947a0-f267-4d55-abc5-73f30f61715a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:14.129387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5947a0-f267-4d55-abc5-73f30f61715a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:14.147900 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 312/629] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Aug 11 09:41:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:14.151162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2523e-af12-4176-bb27-807792891318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:14.151857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2523e-af12-4176-bb27-807792891318 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.152475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2523e-af12-4176-bb27-807792891318 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.152829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2523e-af12-4176-bb27-807792891318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.153257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5947a0-f267-4d55-abc5-73f30f61715a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:14.164990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2523e-af12-4176-bb27-807792891318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:14.166520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2523e-af12-4176-bb27-807792891318 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:14.169518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-3ba2523e-af12-4176-bb27-807792891318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:14.183547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:14.190736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:14.193360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:14.193360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:14.193360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:14.193360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 318/630] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:14.193360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e26aa-5d95-4622-a3fd-1f532d426c1f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.193360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e26aa-5d95-4622-a3fd-1f532d426c1f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.193360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e26aa-5d95-4622-a3fd-1f532d426c1f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.193360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 319/631] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:14.193360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e26aa-5d95-4622-a3fd-1f532d426c1f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:14.251831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:14.267413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5947a0-f267-4d55-abc5-73f30f61715a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:14.302358 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998274e-28f5-4a87-b542-608bf9d72588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:14.302973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998274e-28f5-4a87-b542-608bf9d72588 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.303251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998274e-28f5-4a87-b542-608bf9d72588 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.303526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998274e-28f5-4a87-b542-608bf9d72588 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.304171 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998274e-28f5-4a87-b542-608bf9d72588 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:14.304473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998274e-28f5-4a87-b542-608bf9d72588 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:14.305501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e26aa-5d95-4622-a3fd-1f532d426c1f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:14.305937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998274e-28f5-4a87-b542-608bf9d72588 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:14.322775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 313/632] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:14.322775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998274e-28f5-4a87-b542-608bf9d72588 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:14.347652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582ffe6-a2f4-478d-afd2-c565e46e47cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:14.348309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582ffe6-a2f4-478d-afd2-c565e46e47cf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.348697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582ffe6-a2f4-478d-afd2-c565e46e47cf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.348974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582ffe6-a2f4-478d-afd2-c565e46e47cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.359373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582ffe6-a2f4-478d-afd2-c565e46e47cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:14.360920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582ffe6-a2f4-478d-afd2-c565e46e47cf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b582ffe6-a2f4-478d-afd2-c565e46e47cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:14.376704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:14.377890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:14.379151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:14.379151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:14.379151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:14.379151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 320/633] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:14.379980 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c05151-8899-4d2a-94f7-afe872ef1cff None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.380237 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c05151-8899-4d2a-94f7-afe872ef1cff None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.380512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c05151-8899-4d2a-94f7-afe872ef1cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.431361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998274e-28f5-4a87-b542-608bf9d72588 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:14.469966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee23fb4-8c6b-4b63-90f9-2cb6607ca430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:14.470594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee23fb4-8c6b-4b63-90f9-2cb6607ca430 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.470877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee23fb4-8c6b-4b63-90f9-2cb6607ca430 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.471151 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee23fb4-8c6b-4b63-90f9-2cb6607ca430 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.471816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee23fb4-8c6b-4b63-90f9-2cb6607ca430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:14.472122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee23fb4-8c6b-4b63-90f9-2cb6607ca430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:14.473593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee23fb4-8c6b-4b63-90f9-2cb6607ca430 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:14.487499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 314/634] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:14.490238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee23fb4-8c6b-4b63-90f9-2cb6607ca430 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:14.499559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:14.521667 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a0c05151-8899-4d2a-94f7-afe872ef1cff None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:14.522864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a0c05151-8899-4d2a-94f7-afe872ef1cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:14.568137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:14.590313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 321/635] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:14.591197 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c05151-8899-4d2a-94f7-afe872ef1cff None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:14.602508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee23fb4-8c6b-4b63-90f9-2cb6607ca430 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:14.638722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb91e0-f9b6-43e9-9c90-0bdb99898797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:14.639347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb91e0-f9b6-43e9-9c90-0bdb99898797 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.639625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb91e0-f9b6-43e9-9c90-0bdb99898797 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.639903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb91e0-f9b6-43e9-9c90-0bdb99898797 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.646811 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb91e0-f9b6-43e9-9c90-0bdb99898797 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:14.648319 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb91e0-f9b6-43e9-9c90-0bdb99898797 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-63cb91e0-f9b6-43e9-9c90-0bdb99898797 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:14.659142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:14.660223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:14.661805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:14.661805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:14.661805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:14.661805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:14.661805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:14.661805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:14.661805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:14.661805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:14.661805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 315/636] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:14.661805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb91e0-f9b6-43e9-9c90-0bdb99898797 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:14.694929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c05151-8899-4d2a-94f7-afe872ef1cff None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:14.728184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:14.728902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8fd08-2f00-4df9-9de7-5a22fcc0fbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:14.729538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8fd08-2f00-4df9-9de7-5a22fcc0fbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.729819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8fd08-2f00-4df9-9de7-5a22fcc0fbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.730117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8fd08-2f00-4df9-9de7-5a22fcc0fbb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.730758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8fd08-2f00-4df9-9de7-5a22fcc0fbb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:14.731053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8fd08-2f00-4df9-9de7-5a22fcc0fbb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:14.732394 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8fd08-2f00-4df9-9de7-5a22fcc0fbb2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:14.744090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 322/637] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:14.745329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f608e43-daa0-4662-8cc8-8a9ab92728c6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.745611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f608e43-daa0-4662-8cc8-8a9ab92728c6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.745882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f608e43-daa0-4662-8cc8-8a9ab92728c6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.749067 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 323/638] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:14.749067 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f608e43-daa0-4662-8cc8-8a9ab92728c6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:14.765832 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb91e0-f9b6-43e9-9c90-0bdb99898797 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:14.790069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:14.799832 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94f450-aaa7-4607-85a9-028822855780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:14.800448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94f450-aaa7-4607-85a9-028822855780 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.800742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94f450-aaa7-4607-85a9-028822855780 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.801766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94f450-aaa7-4607-85a9-028822855780 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.808454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94f450-aaa7-4607-85a9-028822855780 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:14.809910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94f450-aaa7-4607-85a9-028822855780 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-1f94f450-aaa7-4607-85a9-028822855780 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:14.823050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:14.828786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 316/639] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:14.830056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94f450-aaa7-4607-85a9-028822855780 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:14.851867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f608e43-daa0-4662-8cc8-8a9ab92728c6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:14.886021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1267c66-fcaa-49ba-b04b-401903ee7432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:14.886647 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1267c66-fcaa-49ba-b04b-401903ee7432 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.886930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1267c66-fcaa-49ba-b04b-401903ee7432 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.887202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1267c66-fcaa-49ba-b04b-401903ee7432 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.897497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1267c66-fcaa-49ba-b04b-401903ee7432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d1267c66-fcaa-49ba-b04b-401903ee7432 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:14.902182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:14.903305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 324/640] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:14.904248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1267c66-fcaa-49ba-b04b-401903ee7432 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:14.928628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94f450-aaa7-4607-85a9-028822855780 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:14.952889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:14.961426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb203897-70f6-4b36-9c31-05edece3209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:14.965689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb203897-70f6-4b36-9c31-05edece3209d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.965689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb203897-70f6-4b36-9c31-05edece3209d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.965689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb203897-70f6-4b36-9c31-05edece3209d None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.965689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb203897-70f6-4b36-9c31-05edece3209d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:14.965689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb203897-70f6-4b36-9c31-05edece3209d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:14.965689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb203897-70f6-4b36-9c31-05edece3209d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:14.977369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 317/641] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:14.978658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83803fb6-b322-458a-8851-74375849ca53 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:14.978994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83803fb6-b322-458a-8851-74375849ca53 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:14.979324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83803fb6-b322-458a-8851-74375849ca53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:14.997142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-83803fb6-b322-458a-8851-74375849ca53 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:14.998333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-83803fb6-b322-458a-8851-74375849ca53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:15.004388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:15.008266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1267c66-fcaa-49ba-b04b-401903ee7432 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:15.042846 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee7e32-0071-4b82-9753-ea4532d1b9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:15.043474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee7e32-0071-4b82-9753-ea4532d1b9fe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:15.043748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee7e32-0071-4b82-9753-ea4532d1b9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:15.044017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee7e32-0071-4b82-9753-ea4532d1b9fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:15.050428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee7e32-0071-4b82-9753-ea4532d1b9fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:15.052489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee7e32-0071-4b82-9753-ea4532d1b9fe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-68ee7e32-0071-4b82-9753-ea4532d1b9fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:15.063349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:15.064411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:15.071071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:15.071071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:15.071071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:15.071071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:15.071071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:15.071071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:15.071071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:15.071071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:15.071071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 325/642] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:15.071071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee7e32-0071-4b82-9753-ea4532d1b9fe None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:15.071807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 318/643] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:15.071807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83803fb6-b322-458a-8851-74375849ca53 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:15.170844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee7e32-0071-4b82-9753-ea4532d1b9fe None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:15.177914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83803fb6-b322-458a-8851-74375849ca53 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:15.194784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:15.217330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de391b55-808e-4326-8f13-8f46001e494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:15.217330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de391b55-808e-4326-8f13-8f46001e494d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:15.217330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de391b55-808e-4326-8f13-8f46001e494d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:15.217330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de391b55-808e-4326-8f13-8f46001e494d None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:15.217750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de391b55-808e-4326-8f13-8f46001e494d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:15.217750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de391b55-808e-4326-8f13-8f46001e494d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:15.217750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de391b55-808e-4326-8f13-8f46001e494d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:15.217988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7884293-8656-4e66-9e59-cf8f36947f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:15.217988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7884293-8656-4e66-9e59-cf8f36947f2e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:15.217988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7884293-8656-4e66-9e59-cf8f36947f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:15.217988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7884293-8656-4e66-9e59-cf8f36947f2e None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:15.218289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7884293-8656-4e66-9e59-cf8f36947f2e None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:15.218289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7884293-8656-4e66-9e59-cf8f36947f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:15.218289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7884293-8656-4e66-9e59-cf8f36947f2e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:15.224179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 319/644] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:15.224999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7884293-8656-4e66-9e59-cf8f36947f2e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:15.232604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 326/645] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Aug 11 09:41:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:15.233891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de391b55-808e-4326-8f13-8f46001e494d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:15.252579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:15.322442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7884293-8656-4e66-9e59-cf8f36947f2e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:15.331857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de391b55-808e-4326-8f13-8f46001e494d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:15.356303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961433a-d8fd-4768-9d10-7259e2f98f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:15.356894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961433a-d8fd-4768-9d10-7259e2f98f9f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:15.357208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961433a-d8fd-4768-9d10-7259e2f98f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:15.357478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961433a-d8fd-4768-9d10-7259e2f98f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:15.358199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961433a-d8fd-4768-9d10-7259e2f98f9f None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=34fa13a87ebe4b789ca1884e5aeb9237) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:15.358508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961433a-d8fd-4768-9d10-7259e2f98f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:15.359914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961433a-d8fd-4768-9d10-7259e2f98f9f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:15.366429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998e8b8-e1ed-4162-92ce-093def4c9077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:15.371533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 320/646] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:15 2022] GET /identity/v3/role_assignments?user.id=34fa13a87ebe4b789ca1884e5aeb9237&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:15.371724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998e8b8-e1ed-4162-92ce-093def4c9077 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:15.371724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998e8b8-e1ed-4162-92ce-093def4c9077 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:15.371905 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961433a-d8fd-4768-9d10-7259e2f98f9f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:15.371987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998e8b8-e1ed-4162-92ce-093def4c9077 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:15.371987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998e8b8-e1ed-4162-92ce-093def4c9077 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:15.371987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998e8b8-e1ed-4162-92ce-093def4c9077 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:15.371987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998e8b8-e1ed-4162-92ce-093def4c9077 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:15.391510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 327/647] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:15.392393 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998e8b8-e1ed-4162-92ce-093def4c9077 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:15.424777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:15.492479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961433a-d8fd-4768-9d10-7259e2f98f9f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:15.494576 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:15.519767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998e8b8-e1ed-4162-92ce-093def4c9077 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:15.529360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd03e6-4a9a-4bbd-b6b9-446d93fd6b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:15.530003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd03e6-4a9a-4bbd-b6b9-446d93fd6b5f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:15.530281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd03e6-4a9a-4bbd-b6b9-446d93fd6b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:15.530545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd03e6-4a9a-4bbd-b6b9-446d93fd6b5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:15.544958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd03e6-4a9a-4bbd-b6b9-446d93fd6b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-69fd03e6-4a9a-4bbd-b6b9-446d93fd6b5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:15.550927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:15.552578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:15.553683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 321/648] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:15.554638 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd03e6-4a9a-4bbd-b6b9-446d93fd6b5f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:15.562364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb38399-2773-420a-ad1e-0bf552f75a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:15.563033 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38399-2773-420a-ad1e-0bf552f75a7f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:15.563386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38399-2773-420a-ad1e-0bf552f75a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:15.563750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38399-2773-420a-ad1e-0bf552f75a7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:15.576482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb38399-2773-420a-ad1e-0bf552f75a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:15.578056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb38399-2773-420a-ad1e-0bf552f75a7f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ebb38399-2773-420a-ad1e-0bf552f75a7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:15.595177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:15.596326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:15.597452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:15.597452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:15.597452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:15.603453 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 328/649] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:15.604446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb38399-2773-420a-ad1e-0bf552f75a7f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:15.663921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd03e6-4a9a-4bbd-b6b9-446d93fd6b5f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:15.665621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:15.699529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5596d0b-0fcb-4b5d-af0f-505cbd799f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:15.700132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5596d0b-0fcb-4b5d-af0f-505cbd799f7a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:15.700410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5596d0b-0fcb-4b5d-af0f-505cbd799f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:15.700681 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5596d0b-0fcb-4b5d-af0f-505cbd799f7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:15.712847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb38399-2773-420a-ad1e-0bf552f75a7f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:15.741910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:15.752723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dff9ff-4fa7-442a-8e88-e15123bd2d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:15.752723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dff9ff-4fa7-442a-8e88-e15123bd2d90 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:15.753428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dff9ff-4fa7-442a-8e88-e15123bd2d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:15.753768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dff9ff-4fa7-442a-8e88-e15123bd2d90 None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/915ee44218064af896020d008ff31957/roles/c6c1ea7ecf3d4e00a138f97d91b14f73` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:15.770069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dff9ff-4fa7-442a-8e88-e15123bd2d90 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=915ee44218064af896020d008ff31957, role_id=c6c1ea7ecf3d4e00a138f97d91b14f73)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:15.771130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dff9ff-4fa7-442a-8e88-e15123bd2d90 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:15.773080 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5596d0b-0fcb-4b5d-af0f-505cbd799f7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:15.774607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5596d0b-0fcb-4b5d-af0f-505cbd799f7a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:15.841051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 329/650] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:41:15 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/915ee44218064af896020d008ff31957/roles/c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:41:15.842375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dff9ff-4fa7-442a-8e88-e15123bd2d90 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:15.884767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 322/651] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:15.884767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5596d0b-0fcb-4b5d-af0f-505cbd799f7a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:15.885612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:16.006865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dff9ff-4fa7-442a-8e88-e15123bd2d90 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:16.010446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5596d0b-0fcb-4b5d-af0f-505cbd799f7a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:16.048679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03aeff-5908-4a45-bec3-a465310ca70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:16.049472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03aeff-5908-4a45-bec3-a465310ca70a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:16.049675 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c294ac7-bbd9-4998-a411-9c84b069fb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:16.049805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03aeff-5908-4a45-bec3-a465310ca70a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:16.050234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e03aeff-5908-4a45-bec3-a465310ca70a None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:16.050336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c294ac7-bbd9-4998-a411-9c84b069fb98 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:16.050509 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c294ac7-bbd9-4998-a411-9c84b069fb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:16.050680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03aeff-5908-4a45-bec3-a465310ca70a None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:16.050777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c294ac7-bbd9-4998-a411-9c84b069fb98 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:16.051808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03aeff-5908-4a45-bec3-a465310ca70a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:16.051914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c294ac7-bbd9-4998-a411-9c84b069fb98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:16.052233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c294ac7-bbd9-4998-a411-9c84b069fb98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:16.053086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e03aeff-5908-4a45-bec3-a465310ca70a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:16.053719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c294ac7-bbd9-4998-a411-9c84b069fb98 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:16.062637 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 330/652] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:16.063553 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03aeff-5908-4a45-bec3-a465310ca70a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:16.068994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 323/653] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:16.070118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c294ac7-bbd9-4998-a411-9c84b069fb98 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:16.127659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:16.141874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:16.190912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e03aeff-5908-4a45-bec3-a465310ca70a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:16.192937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c294ac7-bbd9-4998-a411-9c84b069fb98 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:16.234398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0e8e3-47a5-4a62-8604-48ea66a6063b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:16.235120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0e8e3-47a5-4a62-8604-48ea66a6063b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:16.235568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0e8e3-47a5-4a62-8604-48ea66a6063b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:16.236005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0e8e3-47a5-4a62-8604-48ea66a6063b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:16.238533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e66c-27b1-4454-8df8-4aa5d79efcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:16.239713 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e66c-27b1-4454-8df8-4aa5d79efcf8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:16.240042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e66c-27b1-4454-8df8-4aa5d79efcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:16.240326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e66c-27b1-4454-8df8-4aa5d79efcf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:16.248133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0e8e3-47a5-4a62-8604-48ea66a6063b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:16.249669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0e8e3-47a5-4a62-8604-48ea66a6063b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:16.252757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3e66c-27b1-4454-8df8-4aa5d79efcf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:16.254253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3e66c-27b1-4454-8df8-4aa5d79efcf8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f2b0e8e3-47a5-4a62-8604-48ea66a6063b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:16.265004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:16.266717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:16.268769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:16.268769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:16.268769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:16.268769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 331/654] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:16.268769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0e8e3-47a5-4a62-8604-48ea66a6063b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-c9c3e66c-27b1-4454-8df8-4aa5d79efcf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:16.273978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:16.275303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:16.276167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:16.276167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:16.276167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:16.276167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 324/655] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:16.276167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e66c-27b1-4454-8df8-4aa5d79efcf8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:16.375755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:16.386598 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0e8e3-47a5-4a62-8604-48ea66a6063b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:16.412270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e66c-27b1-4454-8df8-4aa5d79efcf8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:16.429364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:16.431311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2c033-59f3-4e08-a7f2-0b9256747410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:16.432155 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2c033-59f3-4e08-a7f2-0b9256747410 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:16.432724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2c033-59f3-4e08-a7f2-0b9256747410 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:16.433486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2c033-59f3-4e08-a7f2-0b9256747410 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:16.443935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2c033-59f3-4e08-a7f2-0b9256747410 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:16.446240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce2c033-59f3-4e08-a7f2-0b9256747410 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:16.450674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94212d21-3ecc-4bca-b4c4-d4e294292360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:16.451353 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94212d21-3ecc-4bca-b4c4-d4e294292360 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:16.451703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94212d21-3ecc-4bca-b4c4-d4e294292360 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:16.452046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94212d21-3ecc-4bca-b4c4-d4e294292360 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:16.452756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94212d21-3ecc-4bca-b4c4-d4e294292360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:16.453145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94212d21-3ecc-4bca-b4c4-d4e294292360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:16.454563 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94212d21-3ecc-4bca-b4c4-d4e294292360 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0ce2c033-59f3-4e08-a7f2-0b9256747410 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:16.464297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:16.465504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:16.466671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:16.466671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:16.466671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:16.466671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:16.466671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:16.466671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:16.466671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:16.466671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:16.466671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 09:41:16.466671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:16.470335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 332/656] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:16.471334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2c033-59f3-4e08-a7f2-0b9256747410 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:16.480287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 325/657] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:16.480287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94212d21-3ecc-4bca-b4c4-d4e294292360 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:16.624203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94212d21-3ecc-4bca-b4c4-d4e294292360 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:16.638827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce2c033-59f3-4e08-a7f2-0b9256747410 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:16.681040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023a54d-93b4-43da-a922-e999e2864b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:16.681040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023a54d-93b4-43da-a922-e999e2864b64 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:16.681040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023a54d-93b4-43da-a922-e999e2864b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:16.681040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023a54d-93b4-43da-a922-e999e2864b64 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:16.681454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023a54d-93b4-43da-a922-e999e2864b64 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:16.681454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023a54d-93b4-43da-a922-e999e2864b64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:16.685037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023a54d-93b4-43da-a922-e999e2864b64 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:16.688213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169f4c9-0b11-41e7-ae12-dc46e2c66b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:16.688213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169f4c9-0b11-41e7-ae12-dc46e2c66b83 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:16.688213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169f4c9-0b11-41e7-ae12-dc46e2c66b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:16.688213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169f4c9-0b11-41e7-ae12-dc46e2c66b83 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:16.688536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169f4c9-0b11-41e7-ae12-dc46e2c66b83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:16.688536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169f4c9-0b11-41e7-ae12-dc46e2c66b83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:16.688808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169f4c9-0b11-41e7-ae12-dc46e2c66b83 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:16.704661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 333/658] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:16.705634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169f4c9-0b11-41e7-ae12-dc46e2c66b83 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:16.715446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 326/659] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Aug 11 09:41:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:16.716303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023a54d-93b4-43da-a922-e999e2864b64 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:16.741110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:16.770213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:16.884232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169f4c9-0b11-41e7-ae12-dc46e2c66b83 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:16.889308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023a54d-93b4-43da-a922-e999e2864b64 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:16.940771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f6035-2f28-49a9-9263-46da41d87fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:16.942947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f6035-2f28-49a9-9263-46da41d87fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:16.943450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f6035-2f28-49a9-9263-46da41d87fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:16.944124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f6035-2f28-49a9-9263-46da41d87fe9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:16.950540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dec49-0d9a-4274-93dd-146ee8f9bb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:16.951272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dec49-0d9a-4274-93dd-146ee8f9bb67 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:16.951616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dec49-0d9a-4274-93dd-146ee8f9bb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:16.951952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dec49-0d9a-4274-93dd-146ee8f9bb67 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:16.953481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f6035-2f28-49a9-9263-46da41d87fe9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:16.956019 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f6035-2f28-49a9-9263-46da41d87fe9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:16.963990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dec49-0d9a-4274-93dd-146ee8f9bb67 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:16.966291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dec49-0d9a-4274-93dd-146ee8f9bb67 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ed5f6035-2f28-49a9-9263-46da41d87fe9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:16.980116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:16.981258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:16.982162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:16.982162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:16.982162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:16.982162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:16.982162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:16.982162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:16.982162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:16.982162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-108dec49-0d9a-4274-93dd-146ee8f9bb67 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:16.989168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:16.990564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:16.991518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:16.991518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:16.991518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:16.991518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:16.991518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:16.991518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:16.991518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:16.991518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:16.991518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 09:41:16.991518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:16.994194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 334/660] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:16.994676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 327/661] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:16.995793 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f6035-2f28-49a9-9263-46da41d87fe9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:16.996611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dec49-0d9a-4274-93dd-146ee8f9bb67 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:17.136652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:17.155646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:17.168125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dec49-0d9a-4274-93dd-146ee8f9bb67 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:17.181540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f6035-2f28-49a9-9263-46da41d87fe9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:17.213771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3a5ad-3069-452b-9317-58f7c1d061f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:17.214479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3a5ad-3069-452b-9317-58f7c1d061f4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:17.215405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3a5ad-3069-452b-9317-58f7c1d061f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:17.215763 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3a5ad-3069-452b-9317-58f7c1d061f4 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:17.217145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3a5ad-3069-452b-9317-58f7c1d061f4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:17.217728 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3a5ad-3069-452b-9317-58f7c1d061f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:17.220555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3a5ad-3069-452b-9317-58f7c1d061f4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:17.230586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12ef99-3e9a-46e4-9959-4a81e756d7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:17.231284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12ef99-3e9a-46e4-9959-4a81e756d7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:17.231630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12ef99-3e9a-46e4-9959-4a81e756d7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:17.231965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12ef99-3e9a-46e4-9959-4a81e756d7b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:17.245349 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 328/662] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Aug 11 09:41:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:17.246770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3a5ad-3069-452b-9317-58f7c1d061f4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:17.311543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12ef99-3e9a-46e4-9959-4a81e756d7b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:17.313130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12ef99-3e9a-46e4-9959-4a81e756d7b5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:17.376017 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3a5ad-3069-452b-9317-58f7c1d061f4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:17.385549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:17.415320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ac595-137b-46d4-be20-8847046ac84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:17.416001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ac595-137b-46d4-be20-8847046ac84c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:17.416408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ac595-137b-46d4-be20-8847046ac84c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:17.416778 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ac595-137b-46d4-be20-8847046ac84c None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:17.417615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ac595-137b-46d4-be20-8847046ac84c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:17.418043 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ac595-137b-46d4-be20-8847046ac84c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:17.418449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 335/663] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:17.419285 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12ef99-3e9a-46e4-9959-4a81e756d7b5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:17.424416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ac595-137b-46d4-be20-8847046ac84c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:17.444402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 329/664] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:17.445451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ac595-137b-46d4-be20-8847046ac84c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:17.459446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:17.527062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12ef99-3e9a-46e4-9959-4a81e756d7b5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:17.555993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ac595-137b-46d4-be20-8847046ac84c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:17.563559 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfd166-fd5a-4987-b289-57f9154318ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:17.563829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfd166-fd5a-4987-b289-57f9154318ae None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:17.564105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfd166-fd5a-4987-b289-57f9154318ae None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:17.564376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfd166-fd5a-4987-b289-57f9154318ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:17.593238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5a0af-8ea2-4b8c-9719-66241f4a3be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:17.593979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5a0af-8ea2-4b8c-9719-66241f4a3be6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:17.594280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5a0af-8ea2-4b8c-9719-66241f4a3be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:17.594729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5a0af-8ea2-4b8c-9719-66241f4a3be6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:17.605985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5a0af-8ea2-4b8c-9719-66241f4a3be6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:17.607691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5a0af-8ea2-4b8c-9719-66241f4a3be6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:17.608642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-8ab5a0af-8ea2-4b8c-9719-66241f4a3be6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:17.623630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:17.626372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:17.628622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:17.628622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:17.628622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:17.628622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 330/665] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:17.628622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5a0af-8ea2-4b8c-9719-66241f4a3be6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:17.643611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bfd166-fd5a-4987-b289-57f9154318ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:17.645406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bfd166-fd5a-4987-b289-57f9154318ae None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:17.694630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:17.736625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5a0af-8ea2-4b8c-9719-66241f4a3be6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:17.756830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 336/666] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:17.756830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a75543-09c3-4d96-8265-0bccef37ddfe None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:17.756830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a75543-09c3-4d96-8265-0bccef37ddfe None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:17.756830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a75543-09c3-4d96-8265-0bccef37ddfe None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:17.756830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 337/667] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:17.756830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a75543-09c3-4d96-8265-0bccef37ddfe None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:17.786525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4bd23-0985-431b-8d3e-874950da8766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:17.787371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4bd23-0985-431b-8d3e-874950da8766 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:17.787654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4bd23-0985-431b-8d3e-874950da8766 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:17.787929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4bd23-0985-431b-8d3e-874950da8766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:17.788661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4bd23-0985-431b-8d3e-874950da8766 None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=d83448e4c6704f7c8771132bd67eaa1d) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:17.788975 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4bd23-0985-431b-8d3e-874950da8766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:17.790435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4bd23-0985-431b-8d3e-874950da8766 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:17.796117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 331/668] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:17 2022] GET /identity/v3/role_assignments?user.id=d83448e4c6704f7c8771132bd67eaa1d&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:17.796899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4bd23-0985-431b-8d3e-874950da8766 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:17.852377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:17.900031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a75543-09c3-4d96-8265-0bccef37ddfe None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:17.932893 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4bd23-0985-431b-8d3e-874950da8766 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:17.952589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf06df-67f0-4d7f-bc8e-30f4fff0553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:17.953944 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf06df-67f0-4d7f-bc8e-30f4fff0553a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:17.954411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf06df-67f0-4d7f-bc8e-30f4fff0553a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:17.954851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf06df-67f0-4d7f-bc8e-30f4fff0553a None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:17.955689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf06df-67f0-4d7f-bc8e-30f4fff0553a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:17.956764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf06df-67f0-4d7f-bc8e-30f4fff0553a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:17.958830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf06df-67f0-4d7f-bc8e-30f4fff0553a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:17.962780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:17.972733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4603ba-99a3-4e40-97a7-6feee9130fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:17.973997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4603ba-99a3-4e40-97a7-6feee9130fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:17.974443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4603ba-99a3-4e40-97a7-6feee9130fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:17.974730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4603ba-99a3-4e40-97a7-6feee9130fb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:17.975285 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 338/669] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:17.980895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea25740-10d9-4d29-b67f-9a1757155da1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:17.981209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea25740-10d9-4d29-b67f-9a1757155da1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:17.983813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea25740-10d9-4d29-b67f-9a1757155da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:17.991040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4603ba-99a3-4e40-97a7-6feee9130fb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:17.992976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4603ba-99a3-4e40-97a7-6feee9130fb6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:18.001923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6ea25740-10d9-4d29-b67f-9a1757155da1 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:18.003433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6ea25740-10d9-4d29-b67f-9a1757155da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-1e4603ba-99a3-4e40-97a7-6feee9130fb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:18.007992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:18.009125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:18.010209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:18.010209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:18.010209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:18.013838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 332/670] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:18.014990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c8a1e-7334-4ce8-b105-f8ea85d95736 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:18.015279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c8a1e-7334-4ce8-b105-f8ea85d95736 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:18.015551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c8a1e-7334-4ce8-b105-f8ea85d95736 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:18.016798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 333/671] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:18.017515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c8a1e-7334-4ce8-b105-f8ea85d95736 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:18.089108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 339/672] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:18.089108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea25740-10d9-4d29-b67f-9a1757155da1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:18.136757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c8a1e-7334-4ce8-b105-f8ea85d95736 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:18.153929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:18.170999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3e2d1-d761-4fe2-80fc-cfc15676b469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:18.171628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3e2d1-d761-4fe2-80fc-cfc15676b469 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:18.171915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3e2d1-d761-4fe2-80fc-cfc15676b469 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:18.172188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3e2d1-d761-4fe2-80fc-cfc15676b469 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:18.178957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3e2d1-d761-4fe2-80fc-cfc15676b469 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:18.180405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd3e2d1-d761-4fe2-80fc-cfc15676b469 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-bdd3e2d1-d761-4fe2-80fc-cfc15676b469 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:18.196132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:18.197114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:18.198117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:18.198117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:18.198117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:18.198117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:18.198117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:18.198117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:18.198117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:18.198117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:18.198117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 334/673] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:18.198596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13a837-cc76-46d2-851e-cb153e824bda None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:18.199340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13a837-cc76-46d2-851e-cb153e824bda None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:18.199340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13a837-cc76-46d2-851e-cb153e824bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:18.213709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea25740-10d9-4d29-b67f-9a1757155da1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:18.217929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5a13a837-cc76-46d2-851e-cb153e824bda None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:18.219132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5a13a837-cc76-46d2-851e-cb153e824bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:18.250551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094fc3a-6505-468f-92d1-dd8f30483b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:18.251158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094fc3a-6505-468f-92d1-dd8f30483b97 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:18.251679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094fc3a-6505-468f-92d1-dd8f30483b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:18.251679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094fc3a-6505-468f-92d1-dd8f30483b97 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:18.252259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:18.252376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094fc3a-6505-468f-92d1-dd8f30483b97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:18.252621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094fc3a-6505-468f-92d1-dd8f30483b97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:18.254056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094fc3a-6505-468f-92d1-dd8f30483b97 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:18.266266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 340/674] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:18.267105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094fc3a-6505-468f-92d1-dd8f30483b97 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:18.296754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 335/675] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:18.298263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13a837-cc76-46d2-851e-cb153e824bda None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:18.378897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094fc3a-6505-468f-92d1-dd8f30483b97 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:18.404558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:18.417816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d508864-6ffc-4f90-baf1-f3d06a3606ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:18.418535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d508864-6ffc-4f90-baf1-f3d06a3606ce None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:18.418820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d508864-6ffc-4f90-baf1-f3d06a3606ce None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:18.419171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d508864-6ffc-4f90-baf1-f3d06a3606ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:18.421249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13a837-cc76-46d2-851e-cb153e824bda None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:18.430929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d508864-6ffc-4f90-baf1-f3d06a3606ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6d508864-6ffc-4f90-baf1-f3d06a3606ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:18.435766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:18.436688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:18.438017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 341/676] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:18.443353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d508864-6ffc-4f90-baf1-f3d06a3606ce None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:18.462034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f30df7-f909-47ec-bb94-aca8c9fa17fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:18.462642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30df7-f909-47ec-bb94-aca8c9fa17fc None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:18.462924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30df7-f909-47ec-bb94-aca8c9fa17fc None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:18.463191 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30df7-f909-47ec-bb94-aca8c9fa17fc None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:18.463954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f30df7-f909-47ec-bb94-aca8c9fa17fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:18.464194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f30df7-f909-47ec-bb94-aca8c9fa17fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:18.465661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f30df7-f909-47ec-bb94-aca8c9fa17fc None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:18.477619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 336/677] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:18.478407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f30df7-f909-47ec-bb94-aca8c9fa17fc None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:18.508745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:18.561151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d508864-6ffc-4f90-baf1-f3d06a3606ce None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:18.586308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f30df7-f909-47ec-bb94-aca8c9fa17fc None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:18.598116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fd636-9061-4739-b10b-c8fd5bdd7952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:18.598749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fd636-9061-4739-b10b-c8fd5bdd7952 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:18.599025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fd636-9061-4739-b10b-c8fd5bdd7952 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:18.599292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fd636-9061-4739-b10b-c8fd5bdd7952 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:18.606041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fd636-9061-4739-b10b-c8fd5bdd7952 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:18.607461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fd636-9061-4739-b10b-c8fd5bdd7952 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-247fd636-9061-4739-b10b-c8fd5bdd7952 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:18.619775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:18.620679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:18.621581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:18.621581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:18.621581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:18.621581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:18.621581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:18.621581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:18.621581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:18.621581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:18.621581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 342/678] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:18.622616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fd636-9061-4739-b10b-c8fd5bdd7952 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:18.627749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d6425-ee56-4986-99fc-aa6f9e53cd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:18.630330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d6425-ee56-4986-99fc-aa6f9e53cd1b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:18.630678 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d6425-ee56-4986-99fc-aa6f9e53cd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:18.631020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d6425-ee56-4986-99fc-aa6f9e53cd1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:18.641825 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d6425-ee56-4986-99fc-aa6f9e53cd1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-f45d6425-ee56-4986-99fc-aa6f9e53cd1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:18.647472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:18.648609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:18.649670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:18.650780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 337/679] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:18.651392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d6425-ee56-4986-99fc-aa6f9e53cd1b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:18.679948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:18.741419 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fd636-9061-4739-b10b-c8fd5bdd7952 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:18.746374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:18.760818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d6425-ee56-4986-99fc-aa6f9e53cd1b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:18.778680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad27ab-94e4-4459-8096-a0fac31e1652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:18.779456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad27ab-94e4-4459-8096-a0fac31e1652 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:18.779841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad27ab-94e4-4459-8096-a0fac31e1652 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:18.780239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad27ab-94e4-4459-8096-a0fac31e1652 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:18.781049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad27ab-94e4-4459-8096-a0fac31e1652 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:18.781465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad27ab-94e4-4459-8096-a0fac31e1652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:18.782987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad27ab-94e4-4459-8096-a0fac31e1652 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:18.792184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 343/680] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:18.793111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad27ab-94e4-4459-8096-a0fac31e1652 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:18.801091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208219b5-3dd3-400a-a37b-834365d9f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:18.801760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208219b5-3dd3-400a-a37b-834365d9f885 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:18.802034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208219b5-3dd3-400a-a37b-834365d9f885 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:18.802303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208219b5-3dd3-400a-a37b-834365d9f885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:18.803032 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208219b5-3dd3-400a-a37b-834365d9f885 None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=029bb992f9974eec83c0fe7f9fa58826) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:18.803338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208219b5-3dd3-400a-a37b-834365d9f885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:18.804755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208219b5-3dd3-400a-a37b-834365d9f885 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:18.821046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 338/681] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:18 2022] GET /identity/v3/role_assignments?user.id=029bb992f9974eec83c0fe7f9fa58826&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:18.821046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208219b5-3dd3-400a-a37b-834365d9f885 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:18.923462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad27ab-94e4-4459-8096-a0fac31e1652 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:18.927393 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:18.961886 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208219b5-3dd3-400a-a37b-834365d9f885 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:18.981810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e9899-3612-47e4-adf7-b73f8a39cb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:18.982847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e9899-3612-47e4-adf7-b73f8a39cb03 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:18.983245 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e9899-3612-47e4-adf7-b73f8a39cb03 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:18.984044 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e9899-3612-47e4-adf7-b73f8a39cb03 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:18.984919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e9899-3612-47e4-adf7-b73f8a39cb03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:18.985357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e9899-3612-47e4-adf7-b73f8a39cb03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:18.986938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e9899-3612-47e4-adf7-b73f8a39cb03 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:19.003195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 344/682] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:19.005375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e9899-3612-47e4-adf7-b73f8a39cb03 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:19.014145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1b7c5-ad7c-40f3-9742-e1238daf768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:19.015054 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1b7c5-ad7c-40f3-9742-e1238daf768c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:19.016026 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1b7c5-ad7c-40f3-9742-e1238daf768c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:19.016419 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1b7c5-ad7c-40f3-9742-e1238daf768c None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:19.017378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1b7c5-ad7c-40f3-9742-e1238daf768c None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:19.017783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1b7c5-ad7c-40f3-9742-e1238daf768c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:19.019367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1b7c5-ad7c-40f3-9742-e1238daf768c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:19.035188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 339/683] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:19.036370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1b7c5-ad7c-40f3-9742-e1238daf768c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:19.057333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:19.126584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e9899-3612-47e4-adf7-b73f8a39cb03 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:19.165527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9eca3-9080-4b72-99ec-f35037b2e39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:19.166568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9eca3-9080-4b72-99ec-f35037b2e39d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:19.167110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9eca3-9080-4b72-99ec-f35037b2e39d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:19.167589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9eca3-9080-4b72-99ec-f35037b2e39d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:19.179512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9eca3-9080-4b72-99ec-f35037b2e39d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:19.179655 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1b7c5-ad7c-40f3-9742-e1238daf768c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:19.181411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9eca3-9080-4b72-99ec-f35037b2e39d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-aef9eca3-9080-4b72-99ec-f35037b2e39d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:19.203363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:19.204336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:19.205271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:19.205271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:19.205271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:19.207422 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 345/684] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:19.208706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9eca3-9080-4b72-99ec-f35037b2e39d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:19.223671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62703a96-8d3a-4461-9577-7a83f29eec92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:19.224283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62703a96-8d3a-4461-9577-7a83f29eec92 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:19.224560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62703a96-8d3a-4461-9577-7a83f29eec92 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:19.224830 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62703a96-8d3a-4461-9577-7a83f29eec92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:19.226032 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62703a96-8d3a-4461-9577-7a83f29eec92 None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=4fc6c3d1ce8645808c22d1b59b15a100) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:19.226341 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62703a96-8d3a-4461-9577-7a83f29eec92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:19.228057 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62703a96-8d3a-4461-9577-7a83f29eec92 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:19.232605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:19.236465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 340/685] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:19 2022] GET /identity/v3/role_assignments?user.id=4fc6c3d1ce8645808c22d1b59b15a100&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:19.240071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62703a96-8d3a-4461-9577-7a83f29eec92 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:19.397752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:19.401410 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9eca3-9080-4b72-99ec-f35037b2e39d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:19.450422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62703a96-8d3a-4461-9577-7a83f29eec92 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:19.451444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e679c08-ee11-442c-97f0-865c1e53296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:19.452612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679c08-ee11-442c-97f0-865c1e53296d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:19.452891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679c08-ee11-442c-97f0-865c1e53296d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:19.453190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679c08-ee11-442c-97f0-865c1e53296d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:19.467777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e679c08-ee11-442c-97f0-865c1e53296d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:19.469524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e679c08-ee11-442c-97f0-865c1e53296d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0e679c08-ee11-442c-97f0-865c1e53296d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:19.488857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:19.490005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:19.490881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:19.490881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:19.490881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:19.490881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 346/686] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:19.495405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e679c08-ee11-442c-97f0-865c1e53296d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:19.496734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55eccd-13c5-458d-ba76-1ca1aaca6558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:19.497614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55eccd-13c5-458d-ba76-1ca1aaca6558 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:19.498215 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55eccd-13c5-458d-ba76-1ca1aaca6558 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:19.498704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b55eccd-13c5-458d-ba76-1ca1aaca6558 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:19.499719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55eccd-13c5-458d-ba76-1ca1aaca6558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:19.500222 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55eccd-13c5-458d-ba76-1ca1aaca6558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:19.501901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b55eccd-13c5-458d-ba76-1ca1aaca6558 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:19.517286 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 341/687] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:19.527661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55eccd-13c5-458d-ba76-1ca1aaca6558 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:19.576857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:19.617777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e679c08-ee11-442c-97f0-865c1e53296d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:19.656089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b55eccd-13c5-458d-ba76-1ca1aaca6558 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:19.661780 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838625ef-b2e7-41a2-9683-1c5c3f5b2450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:19.662529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838625ef-b2e7-41a2-9683-1c5c3f5b2450 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:19.662905 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838625ef-b2e7-41a2-9683-1c5c3f5b2450 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:19.663279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838625ef-b2e7-41a2-9683-1c5c3f5b2450 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:19.664029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838625ef-b2e7-41a2-9683-1c5c3f5b2450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:19.664430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838625ef-b2e7-41a2-9683-1c5c3f5b2450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:19.665925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838625ef-b2e7-41a2-9683-1c5c3f5b2450 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:19.679052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 347/688] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:19.684781 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838625ef-b2e7-41a2-9683-1c5c3f5b2450 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:19.696924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe558c62-f47f-4ba7-bc60-3937d1645605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:19.697857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe558c62-f47f-4ba7-bc60-3937d1645605 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:19.698284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe558c62-f47f-4ba7-bc60-3937d1645605 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:19.698764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe558c62-f47f-4ba7-bc60-3937d1645605 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:19.706880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe558c62-f47f-4ba7-bc60-3937d1645605 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:19.709139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe558c62-f47f-4ba7-bc60-3937d1645605 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:19.711275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-fe558c62-f47f-4ba7-bc60-3937d1645605 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:19.726516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:19.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:19.728480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:19.728480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:19.728480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:19.728480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:19.728480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:19.728480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:19.728480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:19.728480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:19.728480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:19.728480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:19.728480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 342/689] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Aug 11 09:41:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:19.736414 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe558c62-f47f-4ba7-bc60-3937d1645605 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:19.844860 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838625ef-b2e7-41a2-9683-1c5c3f5b2450 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:19.899497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:19.911517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d4f8e-a6db-408a-a005-868d20073d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:19.912084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d4f8e-a6db-408a-a005-868d20073d19 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:19.912361 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d4f8e-a6db-408a-a005-868d20073d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:19.912631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d4f8e-a6db-408a-a005-868d20073d19 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:19.925189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d4f8e-a6db-408a-a005-868d20073d19 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:19.927259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d4f8e-a6db-408a-a005-868d20073d19 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:19.929920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe558c62-f47f-4ba7-bc60-3937d1645605 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-9b8d4f8e-a6db-408a-a005-868d20073d19 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:19.941286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:19.943516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:19.944554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:19.944554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:19.944554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:19.944554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:19.944554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:19.944554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:19.944554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:19.944554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:19.944554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:19.944554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:19.944554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 348/690] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:19.945701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b5e5d-da2c-4c9f-98ed-b1d59b617eb1 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:19.948708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b5e5d-da2c-4c9f-98ed-b1d59b617eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:19.948708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b5e5d-da2c-4c9f-98ed-b1d59b617eb1 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:19.951311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 349/691] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:19.952095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b5e5d-da2c-4c9f-98ed-b1d59b617eb1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:19.979549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568aaad-1638-44c6-91c0-94115dd049f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:19.980698 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568aaad-1638-44c6-91c0-94115dd049f5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:19.981348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568aaad-1638-44c6-91c0-94115dd049f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:19.981707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568aaad-1638-44c6-91c0-94115dd049f5 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:19.982773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568aaad-1638-44c6-91c0-94115dd049f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:19.983563 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568aaad-1638-44c6-91c0-94115dd049f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:19.985617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568aaad-1638-44c6-91c0-94115dd049f5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:20.010881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 343/692] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Aug 11 09:41:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:20.014263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0f950-dbf7-4304-b1e9-7d79aa9dfe10 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:20.014551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0f950-dbf7-4304-b1e9-7d79aa9dfe10 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:20.014822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0f950-dbf7-4304-b1e9-7d79aa9dfe10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:20.032764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-70f0f950-dbf7-4304-b1e9-7d79aa9dfe10 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:20.033995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-70f0f950-dbf7-4304-b1e9-7d79aa9dfe10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:20.070580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b5e5d-da2c-4c9f-98ed-b1d59b617eb1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:20.087595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:20.106098 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 344/693] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:20.112613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0f950-dbf7-4304-b1e9-7d79aa9dfe10 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:20.112810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8533ae9-cadd-44fc-86d9-4545b33acc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:20.112810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8533ae9-cadd-44fc-86d9-4545b33acc51 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:20.113037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8533ae9-cadd-44fc-86d9-4545b33acc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:20.114096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8533ae9-cadd-44fc-86d9-4545b33acc51 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:20.115212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8533ae9-cadd-44fc-86d9-4545b33acc51 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:20.115617 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8533ae9-cadd-44fc-86d9-4545b33acc51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:20.117153 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8533ae9-cadd-44fc-86d9-4545b33acc51 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:20.137262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 350/694] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Aug 11 09:41:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:20.143049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8533ae9-cadd-44fc-86d9-4545b33acc51 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:20.193457 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:20.250751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0f950-dbf7-4304-b1e9-7d79aa9dfe10 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:20.251467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8533ae9-cadd-44fc-86d9-4545b33acc51 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:20.287102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ff06e-5289-4efe-82a2-b422e8863883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:20.287303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11866c41-e116-4678-9247-f9968473d602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:20.287852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ff06e-5289-4efe-82a2-b422e8863883 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:20.287970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11866c41-e116-4678-9247-f9968473d602 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:20.288135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11866c41-e116-4678-9247-f9968473d602 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:20.288231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ff06e-5289-4efe-82a2-b422e8863883 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:20.288398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11866c41-e116-4678-9247-f9968473d602 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:20.288548 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ff06e-5289-4efe-82a2-b422e8863883 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:20.349391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:20.359837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11866c41-e116-4678-9247-f9968473d602 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:20.360010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ff06e-5289-4efe-82a2-b422e8863883 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:20.361456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11866c41-e116-4678-9247-f9968473d602 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:20.361653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ff06e-5289-4efe-82a2-b422e8863883 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:20.411588 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:20.468644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 345/695] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:20.469539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11866c41-e116-4678-9247-f9968473d602 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:20.473372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 351/696] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:20.474379 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ff06e-5289-4efe-82a2-b422e8863883 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:20.572318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11866c41-e116-4678-9247-f9968473d602 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:20.580102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ff06e-5289-4efe-82a2-b422e8863883 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:20.593393 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:20.609757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4c908-be3b-436f-8a7a-d74228a27864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:20.610516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4c908-be3b-436f-8a7a-d74228a27864 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:20.610800 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4c908-be3b-436f-8a7a-d74228a27864 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:20.611392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4c908-be3b-436f-8a7a-d74228a27864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:20.617669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda0333-1bdc-4485-aee0-d1b8af8b0c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:20.618364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda0333-1bdc-4485-aee0-d1b8af8b0c05 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:20.618706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda0333-1bdc-4485-aee0-d1b8af8b0c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:20.619037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda0333-1bdc-4485-aee0-d1b8af8b0c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:20.623906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4c908-be3b-436f-8a7a-d74228a27864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:20.625370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4c908-be3b-436f-8a7a-d74228a27864 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:20.631066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda0333-1bdc-4485-aee0-d1b8af8b0c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:20.632745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda0333-1bdc-4485-aee0-d1b8af8b0c05 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:20.640770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-bbc4c908-be3b-436f-8a7a-d74228a27864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:20.644530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:20.646331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:20.647420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:20.647420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:20.647420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:20.649166 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 346/697] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:20.650091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4c908-be3b-436f-8a7a-d74228a27864 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1bda0333-1bdc-4485-aee0-d1b8af8b0c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:20.652327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:20.656656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:20.657550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:20.657550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:20.657550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:20.658602 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 352/698] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:20.659526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda0333-1bdc-4485-aee0-d1b8af8b0c05 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:20.778105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4c908-be3b-436f-8a7a-d74228a27864 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:20.780555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda0333-1bdc-4485-aee0-d1b8af8b0c05 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:20.816815 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a14d88-a131-49b1-9482-422bc8e46668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:20.818107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a14d88-a131-49b1-9482-422bc8e46668 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:20.818605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a14d88-a131-49b1-9482-422bc8e46668 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:20.819072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a14d88-a131-49b1-9482-422bc8e46668 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:20.819938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a14d88-a131-49b1-9482-422bc8e46668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:20.820434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a14d88-a131-49b1-9482-422bc8e46668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:20.822312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a14d88-a131-49b1-9482-422bc8e46668 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:20.823840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2da49-8426-4502-8b38-c0d5b32211c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:20.824644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2da49-8426-4502-8b38-c0d5b32211c8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:20.824926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2da49-8426-4502-8b38-c0d5b32211c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:20.825229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2da49-8426-4502-8b38-c0d5b32211c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:20.834865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:20.837655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 353/699] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:20.838681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a14d88-a131-49b1-9482-422bc8e46668 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:20.839580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2da49-8426-4502-8b38-c0d5b32211c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:20.841139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2da49-8426-4502-8b38-c0d5b32211c8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-84f2da49-8426-4502-8b38-c0d5b32211c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:20.863823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:20.864878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:20.865844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:20.865844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:20.865844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:20.869090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 347/700] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:20.870105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2da49-8426-4502-8b38-c0d5b32211c8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:20.929372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:20.961483 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a14d88-a131-49b1-9482-422bc8e46668 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:20.990267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2da49-8426-4502-8b38-c0d5b32211c8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:21.010776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e904b-6cea-449f-bc5f-e3f5d6165780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:21.012029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e904b-6cea-449f-bc5f-e3f5d6165780 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.012390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e904b-6cea-449f-bc5f-e3f5d6165780 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.012868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e904b-6cea-449f-bc5f-e3f5d6165780 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.013701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e904b-6cea-449f-bc5f-e3f5d6165780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:21.014014 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e904b-6cea-449f-bc5f-e3f5d6165780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:21.016301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e904b-6cea-449f-bc5f-e3f5d6165780 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:21.026239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9292176-2827-436f-bf75-f9d1e21aa6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:21.026923 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9292176-2827-436f-bf75-f9d1e21aa6a7 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.027201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9292176-2827-436f-bf75-f9d1e21aa6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.027469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9292176-2827-436f-bf75-f9d1e21aa6a7 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.028100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9292176-2827-436f-bf75-f9d1e21aa6a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:21.029724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9292176-2827-436f-bf75-f9d1e21aa6a7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:21.035026 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 354/701] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:21.036499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e904b-6cea-449f-bc5f-e3f5d6165780 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:21.113047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 348/702] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:41:20 2022] POST /identity/v3/users => generated 254 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:21.113047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9292176-2827-436f-bf75-f9d1e21aa6a7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:21.158150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e904b-6cea-449f-bc5f-e3f5d6165780 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:21.164867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:21.178551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:21.206574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bac71-d2ef-4a11-a8d0-8b0fffd2a70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:21.208744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bac71-d2ef-4a11-a8d0-8b0fffd2a70b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.209528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bac71-d2ef-4a11-a8d0-8b0fffd2a70b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.210003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bac71-d2ef-4a11-a8d0-8b0fffd2a70b None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.211010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bac71-d2ef-4a11-a8d0-8b0fffd2a70b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:21.211572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bac71-d2ef-4a11-a8d0-8b0fffd2a70b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:21.213588 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bac71-d2ef-4a11-a8d0-8b0fffd2a70b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:21.230843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9292176-2827-436f-bf75-f9d1e21aa6a7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:21.235052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 355/703] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:21.235052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da325-ba4a-42de-8097-65fa7147ffb0 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.235052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da325-ba4a-42de-8097-65fa7147ffb0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.235052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166da325-ba4a-42de-8097-65fa7147ffb0 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.235052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 356/704] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:21.236581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166da325-ba4a-42de-8097-65fa7147ffb0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:21.277247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2ca90-7c21-4584-8ea9-a028600ba064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:21.277990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2ca90-7c21-4584-8ea9-a028600ba064 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.278342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2ca90-7c21-4584-8ea9-a028600ba064 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.278726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2ca90-7c21-4584-8ea9-a028600ba064 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.287265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2ca90-7c21-4584-8ea9-a028600ba064 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:21.288590 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2ca90-7c21-4584-8ea9-a028600ba064 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-50f2ca90-7c21-4584-8ea9-a028600ba064 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:21.304758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:21.305866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:21.306732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:21.306732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:21.306732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:21.306732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:21.306732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:21.306732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:21.306732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:21.306732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:21.306732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 349/705] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:21.312401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54544c64-7c85-4cd6-a037-f641f33ab9e7 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.312401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54544c64-7c85-4cd6-a037-f641f33ab9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.312607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54544c64-7c85-4cd6-a037-f641f33ab9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.336877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-54544c64-7c85-4cd6-a037-f641f33ab9e7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:21.338112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-54544c64-7c85-4cd6-a037-f641f33ab9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:21.365553 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166da325-ba4a-42de-8097-65fa7147ffb0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:21.411537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87046d77-4ac4-4dfa-a816-fec814064da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:21.412298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046d77-4ac4-4dfa-a816-fec814064da0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.412653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046d77-4ac4-4dfa-a816-fec814064da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.413061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046d77-4ac4-4dfa-a816-fec814064da0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.419980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 350/706] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:21.421156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54544c64-7c85-4cd6-a037-f641f33ab9e7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:21.427151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87046d77-4ac4-4dfa-a816-fec814064da0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:21.428983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87046d77-4ac4-4dfa-a816-fec814064da0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-87046d77-4ac4-4dfa-a816-fec814064da0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:21.443173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:21.444143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:21.445079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:21.445079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:21.445079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:21.445079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:21.445079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:21.445079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:21.445079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:21.445079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:21.446497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 357/707] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:21.447209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87046d77-4ac4-4dfa-a816-fec814064da0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:21.493309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:21.495413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:21.555342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87046d77-4ac4-4dfa-a816-fec814064da0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:21.567646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54544c64-7c85-4cd6-a037-f641f33ab9e7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:21.591762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bcd66-8380-488b-bcda-95cc8ce12c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:21.592385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bcd66-8380-488b-bcda-95cc8ce12c0a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.592666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bcd66-8380-488b-bcda-95cc8ce12c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.592940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bcd66-8380-488b-bcda-95cc8ce12c0a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.593570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bcd66-8380-488b-bcda-95cc8ce12c0a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:21.594919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bcd66-8380-488b-bcda-95cc8ce12c0a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-726bcd66-8380-488b-bcda-95cc8ce12c0a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:21.606421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:21.607429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:21.608272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 358/708] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Aug 11 09:41:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:21.609057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bcd66-8380-488b-bcda-95cc8ce12c0a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:21.611486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a9970-dd50-44c1-995b-8abd18b7ec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:21.612102 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9970-dd50-44c1-995b-8abd18b7ec05 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.612663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9970-dd50-44c1-995b-8abd18b7ec05 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.612943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9970-dd50-44c1-995b-8abd18b7ec05 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.614513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a9970-dd50-44c1-995b-8abd18b7ec05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:21.614855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a9970-dd50-44c1-995b-8abd18b7ec05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:21.616496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a9970-dd50-44c1-995b-8abd18b7ec05 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:21.636882 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 351/709] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:21.637821 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a9970-dd50-44c1-995b-8abd18b7ec05 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:21.741315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bcd66-8380-488b-bcda-95cc8ce12c0a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:21.742216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:21.743982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:21.748977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a9970-dd50-44c1-995b-8abd18b7ec05 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:21.777747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2772b-e7ab-4ff0-bbae-d41047d85cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:21.778630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2772b-e7ab-4ff0-bbae-d41047d85cfa None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.778908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2772b-e7ab-4ff0-bbae-d41047d85cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.779179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2772b-e7ab-4ff0-bbae-d41047d85cfa None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.779868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2772b-e7ab-4ff0-bbae-d41047d85cfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:21.780168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2772b-e7ab-4ff0-bbae-d41047d85cfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:21.781608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2772b-e7ab-4ff0-bbae-d41047d85cfa None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:21.783569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412c0b-824b-4d9e-9049-ef2ff70c28f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:21.784247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412c0b-824b-4d9e-9049-ef2ff70c28f6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.784587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412c0b-824b-4d9e-9049-ef2ff70c28f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.784924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412c0b-824b-4d9e-9049-ef2ff70c28f6 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.785681 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28412c0b-824b-4d9e-9049-ef2ff70c28f6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:21.786081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28412c0b-824b-4d9e-9049-ef2ff70c28f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:21.787492 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28412c0b-824b-4d9e-9049-ef2ff70c28f6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:21.796680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 352/710] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Aug 11 09:41:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:21.796680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412c0b-824b-4d9e-9049-ef2ff70c28f6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:21.799114 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 359/711] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:21.800317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d60d5d-78d7-4711-a5be-f6c595c8fb26 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.800660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d60d5d-78d7-4711-a5be-f6c595c8fb26 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.801119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d60d5d-78d7-4711-a5be-f6c595c8fb26 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.802420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 360/712] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:21.805914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d60d5d-78d7-4711-a5be-f6c595c8fb26 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:21.913129 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412c0b-824b-4d9e-9049-ef2ff70c28f6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:21.914305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d60d5d-78d7-4711-a5be-f6c595c8fb26 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:21.950306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81319d0-4b6e-40d2-9240-487981dfacac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:21.950680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc48adb-88f0-4138-9065-35835bac233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:21.951174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81319d0-4b6e-40d2-9240-487981dfacac None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.951290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48adb-88f0-4138-9065-35835bac233b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.951599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48adb-88f0-4138-9065-35835bac233b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.951694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81319d0-4b6e-40d2-9240-487981dfacac None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.951874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48adb-88f0-4138-9065-35835bac233b None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.952129 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81319d0-4b6e-40d2-9240-487981dfacac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.952505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc48adb-88f0-4138-9065-35835bac233b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:21.952807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc48adb-88f0-4138-9065-35835bac233b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:21.953155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81319d0-4b6e-40d2-9240-487981dfacac None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=34fa13a87ebe4b789ca1884e5aeb9237) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:21.953554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81319d0-4b6e-40d2-9240-487981dfacac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:21.954225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc48adb-88f0-4138-9065-35835bac233b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:21.955116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81319d0-4b6e-40d2-9240-487981dfacac None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:21.969390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 361/713] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Aug 11 09:41:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:21.969390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc48adb-88f0-4138-9065-35835bac233b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:21.969695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 353/714] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:21 2022] GET /identity/v3/role_assignments?user.id=34fa13a87ebe4b789ca1884e5aeb9237&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:21.969695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b0c4a-ba7b-401c-b741-e2d7844bd8a8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:21.969695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b0c4a-ba7b-401c-b741-e2d7844bd8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:21.969695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b0c4a-ba7b-401c-b741-e2d7844bd8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:21.988706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-514b0c4a-ba7b-401c-b741-e2d7844bd8a8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:21.990146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-514b0c4a-ba7b-401c-b741-e2d7844bd8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:22.018416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:22.058914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:22.109028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 354/715] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:22.155767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b0c4a-ba7b-401c-b741-e2d7844bd8a8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:22.177803 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc48adb-88f0-4138-9065-35835bac233b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:22.229131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218163c-e8a9-4c41-b8cd-342f216a0fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:22.229720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218163c-e8a9-4c41-b8cd-342f216a0fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:22.230002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218163c-e8a9-4c41-b8cd-342f216a0fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:22.230274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218163c-e8a9-4c41-b8cd-342f216a0fb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:22.231008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218163c-e8a9-4c41-b8cd-342f216a0fb0 None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=915ee44218064af896020d008ff31957) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:22.231316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218163c-e8a9-4c41-b8cd-342f216a0fb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:22.232736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218163c-e8a9-4c41-b8cd-342f216a0fb0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:22.239928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 362/716] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:21 2022] GET /identity/v3/role_assignments?user.id=915ee44218064af896020d008ff31957&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 608 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:22.297075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b0c4a-ba7b-401c-b741-e2d7844bd8a8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:22.335887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0eec5-dae1-44b5-b7ef-ba7b6d41d2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:22.336881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0eec5-dae1-44b5-b7ef-ba7b6d41d2de None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:22.337382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0eec5-dae1-44b5-b7ef-ba7b6d41d2de None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:22.337872 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0eec5-dae1-44b5-b7ef-ba7b6d41d2de None admin] PATH_INFO: `/v3/services/network` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:22.338679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0eec5-dae1-44b5-b7ef-ba7b6d41d2de None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:22.340443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0eec5-dae1-44b5-b7ef-ba7b6d41d2de None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-bce0eec5-dae1-44b5-b7ef-ba7b6d41d2de None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:22.352667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:22.353714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 09:41:22.354526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:22.357335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 355/717] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:22 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:22.364362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218163c-e8a9-4c41-b8cd-342f216a0fb0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:22.494696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:22.505189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218163c-e8a9-4c41-b8cd-342f216a0fb0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:22.550375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d5ea3-0769-4c2f-ba0c-41e45dd37bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:22.551168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d5ea3-0769-4c2f-ba0c-41e45dd37bbb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:22.551509 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d5ea3-0769-4c2f-ba0c-41e45dd37bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:22.551901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d5ea3-0769-4c2f-ba0c-41e45dd37bbb None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:22.552543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d5ea3-0769-4c2f-ba0c-41e45dd37bbb None admin] RBAC: Adding query filter params (name=network) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:22.552845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d5ea3-0769-4c2f-ba0c-41e45dd37bbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:22.554338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d5ea3-0769-4c2f-ba0c-41e45dd37bbb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:22.559362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 363/718] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:22.564533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0eec5-dae1-44b5-b7ef-ba7b6d41d2de None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:22.587063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:22.674375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0eec5-dae1-44b5-b7ef-ba7b6d41d2de None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:22.710382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77086f6c-0b0c-454f-b814-7b2c126457af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:22.710995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77086f6c-0b0c-454f-b814-7b2c126457af None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:22.711267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77086f6c-0b0c-454f-b814-7b2c126457af None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:22.711538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77086f6c-0b0c-454f-b814-7b2c126457af None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:22.712169 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77086f6c-0b0c-454f-b814-7b2c126457af None admin] RBAC: Adding query filter params (type=network) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:22.712465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77086f6c-0b0c-454f-b814-7b2c126457af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:22.713868 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77086f6c-0b0c-454f-b814-7b2c126457af None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:22.719844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 356/719] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:22 2022] GET /identity/v3/services?type=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:23.355544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b01c1-3e12-4538-89a2-30f336352dff None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:23.355966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b01c1-3e12-4538-89a2-30f336352dff None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:23.356085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b01c1-3e12-4538-89a2-30f336352dff None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:23.357929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 364/720] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:23.366962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316d2cf-b129-42f6-8fa8-86eac4aedc5f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:23.367480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316d2cf-b129-42f6-8fa8-86eac4aedc5f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:23.367769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316d2cf-b129-42f6-8fa8-86eac4aedc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:23.394165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d316d2cf-b129-42f6-8fa8-86eac4aedc5f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:23.395880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d316d2cf-b129-42f6-8fa8-86eac4aedc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:23.483466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 357/721] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:23.527783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b01c1-3e12-4538-89a2-30f336352dff None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:23.674713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b01c1-3e12-4538-89a2-30f336352dff None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:23.726124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe4c89-0ee4-48c1-8fe6-4c1b072a8e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:23.726763 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe4c89-0ee4-48c1-8fe6-4c1b072a8e40 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:23.727045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe4c89-0ee4-48c1-8fe6-4c1b072a8e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:23.727321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe4c89-0ee4-48c1-8fe6-4c1b072a8e40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:23.743724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe4c89-0ee4-48c1-8fe6-4c1b072a8e40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b2fe4c89-0ee4-48c1-8fe6-4c1b072a8e40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:23.748370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:23.749414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:23.750298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:23.752414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 365/722] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:23.757449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316d2cf-b129-42f6-8fa8-86eac4aedc5f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:23.793366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:24.014717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316d2cf-b129-42f6-8fa8-86eac4aedc5f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:24.055955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e32e79-cedc-4367-be22-57b2b4cad9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:24.056566 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e32e79-cedc-4367-be22-57b2b4cad9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:24.056845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e32e79-cedc-4367-be22-57b2b4cad9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:24.061172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e32e79-cedc-4367-be22-57b2b4cad9b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:24.061867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e32e79-cedc-4367-be22-57b2b4cad9b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:24.062176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e32e79-cedc-4367-be22-57b2b4cad9b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:24.063542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e32e79-cedc-4367-be22-57b2b4cad9b9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:24.073140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 358/723] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:24.078592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe4c89-0ee4-48c1-8fe6-4c1b072a8e40 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:24.109078 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:24.201852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe4c89-0ee4-48c1-8fe6-4c1b072a8e40 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:24.237189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b3797-8674-4517-afee-84f5e47b4ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:24.237834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b3797-8674-4517-afee-84f5e47b4ffa None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:24.238359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b3797-8674-4517-afee-84f5e47b4ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:24.238657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b3797-8674-4517-afee-84f5e47b4ffa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:24.249223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b3797-8674-4517-afee-84f5e47b4ffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:24.250658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b3797-8674-4517-afee-84f5e47b4ffa None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d34b3797-8674-4517-afee-84f5e47b4ffa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:24.266326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:24.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:24.268219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:24.268219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:24.268219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:24.268219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 366/724] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:24.274114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e32e79-cedc-4367-be22-57b2b4cad9b9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:24.339428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2b53-4fc4-4b40-875e-d46b292299d7 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:24.339881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2b53-4fc4-4b40-875e-d46b292299d7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:24.340257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2b53-4fc4-4b40-875e-d46b292299d7 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:24.341988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 367/725] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:24.346770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9013f0a-decd-4986-8a39-56d757ab6e64 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:24.347189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9013f0a-decd-4986-8a39-56d757ab6e64 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:24.347613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9013f0a-decd-4986-8a39-56d757ab6e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:24.374534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:24.398051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e9013f0a-decd-4986-8a39-56d757ab6e64 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:24.399265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e9013f0a-decd-4986-8a39-56d757ab6e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:24.401458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e32e79-cedc-4367-be22-57b2b4cad9b9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:24.436131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d1080-9ec1-4db5-ad33-d41c12521def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:24.436753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d1080-9ec1-4db5-ad33-d41c12521def None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:24.437044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d1080-9ec1-4db5-ad33-d41c12521def None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:24.437322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d1080-9ec1-4db5-ad33-d41c12521def None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:24.437961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d1080-9ec1-4db5-ad33-d41c12521def None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:24.438259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d1080-9ec1-4db5-ad33-d41c12521def None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:24.439610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d1080-9ec1-4db5-ad33-d41c12521def None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:24.457796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 359/726] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:24.459620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d1080-9ec1-4db5-ad33-d41c12521def None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:24.524566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:24.544988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 368/727] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:24.555530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d7a47-0a0b-4b53-bc31-75c24976a8fd None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:24.555809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d7a47-0a0b-4b53-bc31-75c24976a8fd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:24.556116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d7a47-0a0b-4b53-bc31-75c24976a8fd None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:24.557971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 369/728] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:24.566463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7ff2e-2361-4e1a-9ade-be444b8cdc69 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:24.566784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7ff2e-2361-4e1a-9ade-be444b8cdc69 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:24.567019 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7ff2e-2361-4e1a-9ade-be444b8cdc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:24.588429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-64e7ff2e-2361-4e1a-9ade-be444b8cdc69 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:24.589681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-64e7ff2e-2361-4e1a-9ade-be444b8cdc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:24.667715 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 370/729] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 09:41:24.668591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7ff2e-2361-4e1a-9ade-be444b8cdc69 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:24.749903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d1080-9ec1-4db5-ad33-d41c12521def None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:24.787896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668d914-00da-4d99-9cf9-9f9639cfdbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:24.788455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668d914-00da-4d99-9cf9-9f9639cfdbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:24.788732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668d914-00da-4d99-9cf9-9f9639cfdbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:24.789003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668d914-00da-4d99-9cf9-9f9639cfdbe1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:24.791149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:24.797514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668d914-00da-4d99-9cf9-9f9639cfdbe1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:24.799009 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668d914-00da-4d99-9cf9-9f9639cfdbe1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:24.804233 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7ff2e-2361-4e1a-9ade-be444b8cdc69 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9668d914-00da-4d99-9cf9-9f9639cfdbe1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:24.811165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:24.812315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:24.813401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:24.813401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:24.813401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:24.813401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:24.813401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:24.813401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:24.813401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:24.813401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:24.813401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:24.813401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:24.817514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 360/730] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Aug 11 09:41:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:24.818436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668d914-00da-4d99-9cf9-9f9639cfdbe1 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:24.839660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:24.852846 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdb6c4-0993-41bb-b0ce-e6614841c070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:24.854073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdb6c4-0993-41bb-b0ce-e6614841c070 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:24.854927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdb6c4-0993-41bb-b0ce-e6614841c070 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:24.855607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdb6c4-0993-41bb-b0ce-e6614841c070 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:24.869360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdb6c4-0993-41bb-b0ce-e6614841c070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-86bdb6c4-0993-41bb-b0ce-e6614841c070 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:24.875958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:24.877061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:24.878073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:24.882036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 371/731] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:24.883146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdb6c4-0993-41bb-b0ce-e6614841c070 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:24.944856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668d914-00da-4d99-9cf9-9f9639cfdbe1 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:24.986668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eeb4e-f8b2-4db0-b13f-082642dc6051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:24.987554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eeb4e-f8b2-4db0-b13f-082642dc6051 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:24.987985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eeb4e-f8b2-4db0-b13f-082642dc6051 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:24.988531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7eeb4e-f8b2-4db0-b13f-082642dc6051 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:24.989350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7eeb4e-f8b2-4db0-b13f-082642dc6051 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:24.990879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7eeb4e-f8b2-4db0-b13f-082642dc6051 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:25.014874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdb6c4-0993-41bb-b0ce-e6614841c070 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:25.052693 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead3347-4b24-4fdf-82b4-b811a39489a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:25.053262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead3347-4b24-4fdf-82b4-b811a39489a9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.053545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead3347-4b24-4fdf-82b4-b811a39489a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.053820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead3347-4b24-4fdf-82b4-b811a39489a9 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.054555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead3347-4b24-4fdf-82b4-b811a39489a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:25.054868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead3347-4b24-4fdf-82b4-b811a39489a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:25.056339 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead3347-4b24-4fdf-82b4-b811a39489a9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:25.079166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 372/732] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Aug 11 09:41:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:25.079166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead3347-4b24-4fdf-82b4-b811a39489a9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:25.102228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 361/733] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:41:24 2022] POST /identity/v3/services => generated 251 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:25.104419 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eeb4e-f8b2-4db0-b13f-082642dc6051 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:25.141246 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:25.179438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:25.220695 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead3347-4b24-4fdf-82b4-b811a39489a9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:25.266972 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7eeb4e-f8b2-4db0-b13f-082642dc6051 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:25.268143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37875655-4fef-4ebe-bfb6-698e88ee2aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:25.268786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37875655-4fef-4ebe-bfb6-698e88ee2aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.269093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37875655-4fef-4ebe-bfb6-698e88ee2aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.269918 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37875655-4fef-4ebe-bfb6-698e88ee2aa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.270857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37875655-4fef-4ebe-bfb6-698e88ee2aa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:25.271175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37875655-4fef-4ebe-bfb6-698e88ee2aa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:25.273681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37875655-4fef-4ebe-bfb6-698e88ee2aa4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:25.284865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 373/734] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:25.286934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13e404-8e94-4888-8338-053766990afe None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.287224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13e404-8e94-4888-8338-053766990afe None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.287516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13e404-8e94-4888-8338-053766990afe None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.293704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 374/735] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:25.294537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13e404-8e94-4888-8338-053766990afe None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:25.316347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e97406-af25-451c-ad90-bccfd5f6a910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:25.316552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e97406-af25-451c-ad90-bccfd5f6a910 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.317610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e97406-af25-451c-ad90-bccfd5f6a910 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.317918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e97406-af25-451c-ad90-bccfd5f6a910 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.401996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e97406-af25-451c-ad90-bccfd5f6a910 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:25.403496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e97406-af25-451c-ad90-bccfd5f6a910 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:25.421323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13e404-8e94-4888-8338-053766990afe None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:25.429431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:25.454667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:25.459354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66160d52-9f0d-4715-9fe0-3e7959d13677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:25.459994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66160d52-9f0d-4715-9fe0-3e7959d13677 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.460303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66160d52-9f0d-4715-9fe0-3e7959d13677 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.460578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66160d52-9f0d-4715-9fe0-3e7959d13677 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.471143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66160d52-9f0d-4715-9fe0-3e7959d13677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:25.472623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66160d52-9f0d-4715-9fe0-3e7959d13677 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-66160d52-9f0d-4715-9fe0-3e7959d13677 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:25.488578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:25.489782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:25.490755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:25.490755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:25.490755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:25.490755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 375/736] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:25.491073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7aa8d8-4233-470b-bb4c-73f40ae2c252 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.491324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7aa8d8-4233-470b-bb4c-73f40ae2c252 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.491597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7aa8d8-4233-470b-bb4c-73f40ae2c252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.510720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-db7aa8d8-4233-470b-bb4c-73f40ae2c252 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:25.511919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-db7aa8d8-4233-470b-bb4c-73f40ae2c252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:25.518551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 362/737] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:25 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 09:41:25.518551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29e6c-5652-4a58-8677-a37ce3e248e7 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.518551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29e6c-5652-4a58-8677-a37ce3e248e7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.518551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e29e6c-5652-4a58-8677-a37ce3e248e7 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.518551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 363/738] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:25.518551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e29e6c-5652-4a58-8677-a37ce3e248e7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:25.595262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 376/739] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:25.596166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7aa8d8-4233-470b-bb4c-73f40ae2c252 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:25.641856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e29e6c-5652-4a58-8677-a37ce3e248e7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:25.682853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899ba22-a75b-4127-9d82-e46cefe75e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:25.683879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899ba22-a75b-4127-9d82-e46cefe75e5d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.684158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899ba22-a75b-4127-9d82-e46cefe75e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.684429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899ba22-a75b-4127-9d82-e46cefe75e5d None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.685462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899ba22-a75b-4127-9d82-e46cefe75e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:25.685769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899ba22-a75b-4127-9d82-e46cefe75e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:25.687441 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899ba22-a75b-4127-9d82-e46cefe75e5d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:25.698483 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:25.699860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 364/740] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:25.701072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79adabe-34e1-4675-9c1b-b82585f9af8e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.701358 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79adabe-34e1-4675-9c1b-b82585f9af8e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.701632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79adabe-34e1-4675-9c1b-b82585f9af8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.715162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7aa8d8-4233-470b-bb4c-73f40ae2c252 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:25.719498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e79adabe-34e1-4675-9c1b-b82585f9af8e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:25.720685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e79adabe-34e1-4675-9c1b-b82585f9af8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:25.723686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:25.753160 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9db04-9e10-426e-9aa0-b7a3f5811a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:25.754402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9db04-9e10-426e-9aa0-b7a3f5811a69 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.754916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9db04-9e10-426e-9aa0-b7a3f5811a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.755318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9db04-9e10-426e-9aa0-b7a3f5811a69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.767561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9db04-9e10-426e-9aa0-b7a3f5811a69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:25.768989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9db04-9e10-426e-9aa0-b7a3f5811a69 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-13b9db04-9e10-426e-9aa0-b7a3f5811a69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:25.784615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:25.785872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:25.786729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:25.786729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:25.786729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:25.786729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 377/741] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:25.786729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9db04-9e10-426e-9aa0-b7a3f5811a69 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:25.795339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 365/742] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 09:41:25.796252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79adabe-34e1-4675-9c1b-b82585f9af8e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:25.902400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9db04-9e10-426e-9aa0-b7a3f5811a69 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:25.902881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79adabe-34e1-4675-9c1b-b82585f9af8e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:25.938024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2dfea-95f3-4d01-be20-4f72b0c0a38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:25.938234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72567949-829f-4387-ab66-248686fdb463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:25.938656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2dfea-95f3-4d01-be20-4f72b0c0a38f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.938777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567949-829f-4387-ab66-248686fdb463 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:25.938964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567949-829f-4387-ab66-248686fdb463 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.939069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2dfea-95f3-4d01-be20-4f72b0c0a38f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:25.939234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72567949-829f-4387-ab66-248686fdb463 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.939332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2dfea-95f3-4d01-be20-4f72b0c0a38f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:25.946677 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72567949-829f-4387-ab66-248686fdb463 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:25.947953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:25.948115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72567949-829f-4387-ab66-248686fdb463 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:25.950714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:25.952428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2dfea-95f3-4d01-be20-4f72b0c0a38f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-63b2dfea-95f3-4d01-be20-4f72b0c0a38f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:25.957649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:25.959137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:25.960376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-72567949-829f-4387-ab66-248686fdb463 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:25.961004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:25.961865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:25.964086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:25.964086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:25.964086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:25.964086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:25.964086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:25.964086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:25.964086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:25.964086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:25.964086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:25.964086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:25.965420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 378/743] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:25.965420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2dfea-95f3-4d01-be20-4f72b0c0a38f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:25.966545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 366/744] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Aug 11 09:41:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:25.967381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72567949-829f-4387-ab66-248686fdb463 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:26.066882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2dfea-95f3-4d01-be20-4f72b0c0a38f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:26.072646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72567949-829f-4387-ab66-248686fdb463 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:26.100683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dec39-4297-464f-baa4-fd603763f32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:26.101312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dec39-4297-464f-baa4-fd603763f32d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:26.101589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dec39-4297-464f-baa4-fd603763f32d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:26.101864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dec39-4297-464f-baa4-fd603763f32d None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:26.102497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dec39-4297-464f-baa4-fd603763f32d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:26.102800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dec39-4297-464f-baa4-fd603763f32d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:26.104147 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dec39-4297-464f-baa4-fd603763f32d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:26.110692 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc6997-e4c8-415b-8bad-9ba4a5eec18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:26.111491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc6997-e4c8-415b-8bad-9ba4a5eec18e None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:26.111923 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc6997-e4c8-415b-8bad-9ba4a5eec18e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:26.112495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc6997-e4c8-415b-8bad-9ba4a5eec18e None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:26.113241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc6997-e4c8-415b-8bad-9ba4a5eec18e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:26.114905 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc6997-e4c8-415b-8bad-9ba4a5eec18e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:26.120102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 379/745] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:26.120102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dec39-4297-464f-baa4-fd603763f32d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:26.177777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:26.198253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 367/746] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:41:25 2022] POST /identity/v3/services => generated 240 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:26.199137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc6997-e4c8-415b-8bad-9ba4a5eec18e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:26.241073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dec39-4297-464f-baa4-fd603763f32d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:26.279214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:26.303695 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325d691-a6be-4537-a9c9-530df7676fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:26.304477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325d691-a6be-4537-a9c9-530df7676fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:26.304781 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325d691-a6be-4537-a9c9-530df7676fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:26.305193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325d691-a6be-4537-a9c9-530df7676fc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:26.306002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325d691-a6be-4537-a9c9-530df7676fc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:26.306354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325d691-a6be-4537-a9c9-530df7676fc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:26.308052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325d691-a6be-4537-a9c9-530df7676fc8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:26.319270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 380/747] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:26.320423 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325d691-a6be-4537-a9c9-530df7676fc8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:26.338226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc6997-e4c8-415b-8bad-9ba4a5eec18e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:26.378109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e01af-edab-4221-a96c-52f94350de75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:26.379669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e01af-edab-4221-a96c-52f94350de75 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:26.380543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e01af-edab-4221-a96c-52f94350de75 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:26.380997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e01af-edab-4221-a96c-52f94350de75 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:26.382070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e01af-edab-4221-a96c-52f94350de75 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:26.382580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e01af-edab-4221-a96c-52f94350de75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:26.384308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e01af-edab-4221-a96c-52f94350de75 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:26.404339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 368/748] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Aug 11 09:41:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:26.405969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e01af-edab-4221-a96c-52f94350de75 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:26.449120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325d691-a6be-4537-a9c9-530df7676fc8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:26.473402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:26.493707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba67969-4ee1-4923-9f29-6669cabaa04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:26.494612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba67969-4ee1-4923-9f29-6669cabaa04b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:26.494887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba67969-4ee1-4923-9f29-6669cabaa04b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:26.495155 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba67969-4ee1-4923-9f29-6669cabaa04b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:26.504513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba67969-4ee1-4923-9f29-6669cabaa04b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:26.506068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba67969-4ee1-4923-9f29-6669cabaa04b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1ba67969-4ee1-4923-9f29-6669cabaa04b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:26.520473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:26.521414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:26.522278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:26.522278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:26.522278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:26.522278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:26.522278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:26.522278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:26.522278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:26.522278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:26.524552 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 381/749] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:26.531802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba67969-4ee1-4923-9f29-6669cabaa04b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:26.541314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e01af-edab-4221-a96c-52f94350de75 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:26.564295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:26.585884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f93e9e-6724-41a9-a3fa-223a5303fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:26.586958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f93e9e-6724-41a9-a3fa-223a5303fac6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:26.587237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f93e9e-6724-41a9-a3fa-223a5303fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:26.587523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f93e9e-6724-41a9-a3fa-223a5303fac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:26.605489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f93e9e-6724-41a9-a3fa-223a5303fac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:26.606928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f93e9e-6724-41a9-a3fa-223a5303fac6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-c9f93e9e-6724-41a9-a3fa-223a5303fac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:26.621889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:26.622860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:26.623733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:26.623733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:26.623733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:26.627880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 369/750] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:26.635295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f93e9e-6724-41a9-a3fa-223a5303fac6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:26.672840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba67969-4ee1-4923-9f29-6669cabaa04b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:26.714987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1eb2e-74be-4b04-a84f-7d5fa93564a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:26.715440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1eb2e-74be-4b04-a84f-7d5fa93564a2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:26.715725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1eb2e-74be-4b04-a84f-7d5fa93564a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:26.715998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1eb2e-74be-4b04-a84f-7d5fa93564a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:26.728138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1eb2e-74be-4b04-a84f-7d5fa93564a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:26.729644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1eb2e-74be-4b04-a84f-7d5fa93564a2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-41e1eb2e-74be-4b04-a84f-7d5fa93564a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:26.744786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:26.745811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:26.746714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:26.746714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:26.746714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:26.750755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 382/751] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:26.751592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1eb2e-74be-4b04-a84f-7d5fa93564a2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:26.761890 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f93e9e-6724-41a9-a3fa-223a5303fac6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:26.785706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:26.808086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b9ff0-ec1b-41e3-aef2-f023b8409440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:26.808679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b9ff0-ec1b-41e3-aef2-f023b8409440 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:26.808962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b9ff0-ec1b-41e3-aef2-f023b8409440 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:26.809351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b9ff0-ec1b-41e3-aef2-f023b8409440 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:26.809945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b9ff0-ec1b-41e3-aef2-f023b8409440 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:26.810249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b9ff0-ec1b-41e3-aef2-f023b8409440 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:26.811664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b9ff0-ec1b-41e3-aef2-f023b8409440 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:26.829286 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 370/752] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:26.829286 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b9ff0-ec1b-41e3-aef2-f023b8409440 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:26.863048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:26.874965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1eb2e-74be-4b04-a84f-7d5fa93564a2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:26.913451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffca35c-37b9-41ef-b034-6fb1d2fa3381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:26.914463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffca35c-37b9-41ef-b034-6fb1d2fa3381 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:26.914945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffca35c-37b9-41ef-b034-6fb1d2fa3381 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:26.915475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffca35c-37b9-41ef-b034-6fb1d2fa3381 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:26.916330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffca35c-37b9-41ef-b034-6fb1d2fa3381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:26.916898 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffca35c-37b9-41ef-b034-6fb1d2fa3381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:26.918492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffca35c-37b9-41ef-b034-6fb1d2fa3381 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:26.932244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 383/753] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:26.933517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffca35c-37b9-41ef-b034-6fb1d2fa3381 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:26.940076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b9ff0-ec1b-41e3-aef2-f023b8409440 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:26.979268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3b2c0-7d15-4745-a352-302b10b6dc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:26.979802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3b2c0-7d15-4745-a352-302b10b6dc24 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:26.980087 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3b2c0-7d15-4745-a352-302b10b6dc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:26.980420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3b2c0-7d15-4745-a352-302b10b6dc24 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:26.981415 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3b2c0-7d15-4745-a352-302b10b6dc24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:26.981598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3b2c0-7d15-4745-a352-302b10b6dc24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:26.983167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3b2c0-7d15-4745-a352-302b10b6dc24 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:26.996353 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 371/754] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:26.997431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3b2c0-7d15-4745-a352-302b10b6dc24 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:27.035555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:27.046065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffca35c-37b9-41ef-b034-6fb1d2fa3381 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:27.086100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4389698-bae8-4899-82b7-19a1ab1d3405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:27.087156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4389698-bae8-4899-82b7-19a1ab1d3405 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:27.087638 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4389698-bae8-4899-82b7-19a1ab1d3405 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:27.088111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4389698-bae8-4899-82b7-19a1ab1d3405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:27.089185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4389698-bae8-4899-82b7-19a1ab1d3405 None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=813d8cff9d7d49d5995bfc272fee88f3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:27.089719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4389698-bae8-4899-82b7-19a1ab1d3405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:27.091400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4389698-bae8-4899-82b7-19a1ab1d3405 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:27.103177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 384/755] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:26 2022] GET /identity/v3/role_assignments?user.id=813d8cff9d7d49d5995bfc272fee88f3&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:27.103787 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:27.105631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4389698-bae8-4899-82b7-19a1ab1d3405 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:27.110539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3b2c0-7d15-4745-a352-302b10b6dc24 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:27.166640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4523bd-417e-404b-a38d-861c580680bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:27.167646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4523bd-417e-404b-a38d-861c580680bd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:27.167759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4523bd-417e-404b-a38d-861c580680bd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:27.167984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4523bd-417e-404b-a38d-861c580680bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:27.183811 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4523bd-417e-404b-a38d-861c580680bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:27.184031 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4523bd-417e-404b-a38d-861c580680bd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-6e4523bd-417e-404b-a38d-861c580680bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:27.203111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:27.204223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:27.205185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:27.205185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:27.205185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:27.205185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:27.205185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:27.205185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:27.205185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:27.205185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:27.205185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:27.205185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:27.209259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 372/756] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:27.214791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4523bd-417e-404b-a38d-861c580680bd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:27.272964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4389698-bae8-4899-82b7-19a1ab1d3405 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:27.311910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf647c36-496a-455d-aa9a-8ea1020ac60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:27.312649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647c36-496a-455d-aa9a-8ea1020ac60c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:27.312995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647c36-496a-455d-aa9a-8ea1020ac60c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:27.313378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647c36-496a-455d-aa9a-8ea1020ac60c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:27.322695 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf647c36-496a-455d-aa9a-8ea1020ac60c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:27.324239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf647c36-496a-455d-aa9a-8ea1020ac60c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-bf647c36-496a-455d-aa9a-8ea1020ac60c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:27.338279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:27.339199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:27.340056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:27.340056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:27.340056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:27.340056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:27.340056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:27.340056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:27.340056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:27.340056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:27.340056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 385/757] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:27.341327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eca3a9-25b4-4dfe-b224-19252995502c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:27.341707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eca3a9-25b4-4dfe-b224-19252995502c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:27.342078 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eca3a9-25b4-4dfe-b224-19252995502c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:27.343633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 386/758] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:27.348619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4523bd-417e-404b-a38d-861c580680bd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:27.349000 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eca3a9-25b4-4dfe-b224-19252995502c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:27.367485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:27.395525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60930894-c0f3-4cc0-b4c6-e04d1a775c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:27.396408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60930894-c0f3-4cc0-b4c6-e04d1a775c9a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:27.396846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60930894-c0f3-4cc0-b4c6-e04d1a775c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:27.397663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60930894-c0f3-4cc0-b4c6-e04d1a775c9a None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:27.398571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60930894-c0f3-4cc0-b4c6-e04d1a775c9a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:27.399357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60930894-c0f3-4cc0-b4c6-e04d1a775c9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:27.401277 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60930894-c0f3-4cc0-b4c6-e04d1a775c9a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:27.429330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 373/759] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Aug 11 09:41:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:27.432182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aed08a-c927-45a5-b8b9-60cbb0f70934 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:27.432617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aed08a-c927-45a5-b8b9-60cbb0f70934 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:27.433075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aed08a-c927-45a5-b8b9-60cbb0f70934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:27.459429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-01aed08a-c927-45a5-b8b9-60cbb0f70934 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:27.460665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-01aed08a-c927-45a5-b8b9-60cbb0f70934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:27.471537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eca3a9-25b4-4dfe-b224-19252995502c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:27.472059 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:27.517733 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ee183-6f21-42d0-a40f-d10c96905f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:27.518353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ee183-6f21-42d0-a40f-d10c96905f69 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:27.518635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ee183-6f21-42d0-a40f-d10c96905f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:27.518907 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ee183-6f21-42d0-a40f-d10c96905f69 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:27.519667 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ee183-6f21-42d0-a40f-d10c96905f69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:27.519972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ee183-6f21-42d0-a40f-d10c96905f69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:27.522002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ee183-6f21-42d0-a40f-d10c96905f69 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:27.535130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 374/760] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:27.536399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aed08a-c927-45a5-b8b9-60cbb0f70934 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:27.536918 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 387/761] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:27.546037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ee183-6f21-42d0-a40f-d10c96905f69 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:27.652025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aed08a-c927-45a5-b8b9-60cbb0f70934 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:27.656985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:27.687666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ee183-6f21-42d0-a40f-d10c96905f69 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:27.690624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01bb48-79a7-4da6-a9db-791a2aaebc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:27.691803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01bb48-79a7-4da6-a9db-791a2aaebc43 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:27.692561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01bb48-79a7-4da6-a9db-791a2aaebc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:27.693744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01bb48-79a7-4da6-a9db-791a2aaebc43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:27.701363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:27.729191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3ca5-c6e9-4df9-b222-ba92be06616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:27.730307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3ca5-c6e9-4df9-b222-ba92be06616c None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:27.730754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3ca5-c6e9-4df9-b222-ba92be06616c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:27.731222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3ca5-c6e9-4df9-b222-ba92be06616c None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/34fa13a87ebe4b789ca1884e5aeb9237/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:27.733352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d3ca5-c6e9-4df9-b222-ba92be06616c None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=34fa13a87ebe4b789ca1884e5aeb9237, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:27.734952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d3ca5-c6e9-4df9-b222-ba92be06616c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:27.764304 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01bb48-79a7-4da6-a9db-791a2aaebc43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:27.765780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01bb48-79a7-4da6-a9db-791a2aaebc43 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:27.808468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 388/762] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:41:27 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/34fa13a87ebe4b789ca1884e5aeb9237/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:41:27.810303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3ca5-c6e9-4df9-b222-ba92be06616c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:27.939327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 375/763] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:41:27.944705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01bb48-79a7-4da6-a9db-791a2aaebc43 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:27.947618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3ca5-c6e9-4df9-b222-ba92be06616c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:27.992192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593de638-fa1d-41a8-95ff-52b5f296a6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:27.993450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593de638-fa1d-41a8-95ff-52b5f296a6dd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:27.994126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593de638-fa1d-41a8-95ff-52b5f296a6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:27.994583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593de638-fa1d-41a8-95ff-52b5f296a6dd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:27.995463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593de638-fa1d-41a8-95ff-52b5f296a6dd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:27.999249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593de638-fa1d-41a8-95ff-52b5f296a6dd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:28.001462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-593de638-fa1d-41a8-95ff-52b5f296a6dd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:28.011931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:28.018120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:28.020214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 389/764] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Aug 11 09:41:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:28.021377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:28.021455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593de638-fa1d-41a8-95ff-52b5f296a6dd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:28.080102 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01bb48-79a7-4da6-a9db-791a2aaebc43 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:28.118894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44900bfb-ea48-4d65-82af-485956335acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:28.119876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44900bfb-ea48-4d65-82af-485956335acf None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:28.120363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44900bfb-ea48-4d65-82af-485956335acf None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:28.120841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44900bfb-ea48-4d65-82af-485956335acf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:28.129440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593de638-fa1d-41a8-95ff-52b5f296a6dd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:28.132874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44900bfb-ea48-4d65-82af-485956335acf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:28.134571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44900bfb-ea48-4d65-82af-485956335acf None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-44900bfb-ea48-4d65-82af-485956335acf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:28.152920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:28.153933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:28.154843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:28.154843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:28.154843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:28.157576 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 376/765] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:28.163037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44900bfb-ea48-4d65-82af-485956335acf None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:28.167945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51952e-dbdf-44af-9938-a37eef342ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:28.168723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51952e-dbdf-44af-9938-a37eef342ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:28.169129 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51952e-dbdf-44af-9938-a37eef342ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:28.169502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51952e-dbdf-44af-9938-a37eef342ec9 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:28.170239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51952e-dbdf-44af-9938-a37eef342ec9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:28.170640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51952e-dbdf-44af-9938-a37eef342ec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:28.172074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51952e-dbdf-44af-9938-a37eef342ec9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:28.178056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 390/766] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Aug 11 09:41:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:28.183769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51952e-dbdf-44af-9938-a37eef342ec9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:28.252613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:28.293401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:28.303100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44900bfb-ea48-4d65-82af-485956335acf None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:28.309582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51952e-dbdf-44af-9938-a37eef342ec9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:28.371145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4743eb-fa97-4f51-a8ec-bf121f07a897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:28.371853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4743eb-fa97-4f51-a8ec-bf121f07a897 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:28.372319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4743eb-fa97-4f51-a8ec-bf121f07a897 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:28.372684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4743eb-fa97-4f51-a8ec-bf121f07a897 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:28.373496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4743eb-fa97-4f51-a8ec-bf121f07a897 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:28.373990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4743eb-fa97-4f51-a8ec-bf121f07a897 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:28.374675 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f697d4-ee0d-4a5a-9e43-20cc8f2352a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:28.375448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f697d4-ee0d-4a5a-9e43-20cc8f2352a8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:28.375664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4743eb-fa97-4f51-a8ec-bf121f07a897 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:28.376031 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f697d4-ee0d-4a5a-9e43-20cc8f2352a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:28.376460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f697d4-ee0d-4a5a-9e43-20cc8f2352a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:28.377274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f697d4-ee0d-4a5a-9e43-20cc8f2352a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:28.377718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f697d4-ee0d-4a5a-9e43-20cc8f2352a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:28.379259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f697d4-ee0d-4a5a-9e43-20cc8f2352a8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:28.389039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 391/767] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Aug 11 09:41:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:28.389039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4743eb-fa97-4f51-a8ec-bf121f07a897 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:28.408099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 377/768] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:28.414437 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f697d4-ee0d-4a5a-9e43-20cc8f2352a8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:28.554542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f697d4-ee0d-4a5a-9e43-20cc8f2352a8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:28.584087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4743eb-fa97-4f51-a8ec-bf121f07a897 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:28.591079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0478005-11de-48dd-8b6a-d367818d90c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:28.591838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0478005-11de-48dd-8b6a-d367818d90c9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:28.592360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0478005-11de-48dd-8b6a-d367818d90c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:28.592726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0478005-11de-48dd-8b6a-d367818d90c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:28.600115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0478005-11de-48dd-8b6a-d367818d90c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:28.601840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0478005-11de-48dd-8b6a-d367818d90c9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b0478005-11de-48dd-8b6a-d367818d90c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:28.614312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:28.615432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:28.616433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:28.616433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:28.616433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:28.616433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:28.616433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:28.616433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:28.616433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:28.616433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:28.621305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 378/769] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:28.621305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201f7de-7056-4c0e-9db3-c55d5c6dedf8 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:28.621305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201f7de-7056-4c0e-9db3-c55d5c6dedf8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:28.621305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201f7de-7056-4c0e-9db3-c55d5c6dedf8 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:28.621305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 379/770] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:28.624469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201f7de-7056-4c0e-9db3-c55d5c6dedf8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:28.630752 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:28.638665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:28.643850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d3b0f-d633-4540-863f-66302a8beb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:28.645547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d3b0f-d633-4540-863f-66302a8beb67 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:28.649366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d3b0f-d633-4540-863f-66302a8beb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:28.649366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d3b0f-d633-4540-863f-66302a8beb67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:28.649366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d3b0f-d633-4540-863f-66302a8beb67 None admin] RBAC: Adding query filter params (interface=public, service_id=10aee8eb96c44a9a89cf3c270cab92ec, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:28.649366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d3b0f-d633-4540-863f-66302a8beb67 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:28.649366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d3b0f-d633-4540-863f-66302a8beb67 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:28.657509 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 392/771] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Aug 11 09:41:28 2022] GET /identity/v3/endpoints?service_id=10aee8eb96c44a9a89cf3c270cab92ec&interface=public®ion_id=RegionOne => generated 194 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:28.658939 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b05f6-2b1b-4eff-889a-e34b0148b3b8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:28.659556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b05f6-2b1b-4eff-889a-e34b0148b3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:28.660161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b05f6-2b1b-4eff-889a-e34b0148b3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:28.681941 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-592b05f6-2b1b-4eff-889a-e34b0148b3b8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:28.683766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-592b05f6-2b1b-4eff-889a-e34b0148b3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:28.747565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201f7de-7056-4c0e-9db3-c55d5c6dedf8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:28.782517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963899b6-9cd9-4837-9eca-0493ffb5a86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:28.783217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963899b6-9cd9-4837-9eca-0493ffb5a86c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:28.783495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963899b6-9cd9-4837-9eca-0493ffb5a86c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:28.783766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963899b6-9cd9-4837-9eca-0493ffb5a86c None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:28.784454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963899b6-9cd9-4837-9eca-0493ffb5a86c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:28.784708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 393/772] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:28.785182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963899b6-9cd9-4837-9eca-0493ffb5a86c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:28.786076 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b05f6-2b1b-4eff-889a-e34b0148b3b8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:28.786275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963899b6-9cd9-4837-9eca-0493ffb5a86c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:28.800022 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 380/773] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:28.810108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963899b6-9cd9-4837-9eca-0493ffb5a86c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:28.884985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:28.902146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b05f6-2b1b-4eff-889a-e34b0148b3b8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:28.918934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963899b6-9cd9-4837-9eca-0493ffb5a86c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:28.924958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:28.943064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68e24d-61c2-4fb6-bf6c-fb8efcb3bf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:28.943911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68e24d-61c2-4fb6-bf6c-fb8efcb3bf46 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:28.944344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68e24d-61c2-4fb6-bf6c-fb8efcb3bf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:28.944791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68e24d-61c2-4fb6-bf6c-fb8efcb3bf46 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:28.945786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68e24d-61c2-4fb6-bf6c-fb8efcb3bf46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:28.947318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68e24d-61c2-4fb6-bf6c-fb8efcb3bf46 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:28.954156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 394/774] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Aug 11 09:41:28 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:28.955245 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68e24d-61c2-4fb6-bf6c-fb8efcb3bf46 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:28.966807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed6290-81b5-42f8-9d1a-6648b8d901fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:28.967486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed6290-81b5-42f8-9d1a-6648b8d901fa None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:28.967768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed6290-81b5-42f8-9d1a-6648b8d901fa None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:28.968040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed6290-81b5-42f8-9d1a-6648b8d901fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:28.968835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed6290-81b5-42f8-9d1a-6648b8d901fa None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=915ee44218064af896020d008ff31957) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:28.969436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed6290-81b5-42f8-9d1a-6648b8d901fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:28.971003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed6290-81b5-42f8-9d1a-6648b8d901fa None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:28.983749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 381/775] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:28 2022] GET /identity/v3/role_assignments?user.id=915ee44218064af896020d008ff31957&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:29.082007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68e24d-61c2-4fb6-bf6c-fb8efcb3bf46 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:29.123592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57accf-a896-490e-8049-d8ac15669541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:29.124631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57accf-a896-490e-8049-d8ac15669541 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:29.125120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57accf-a896-490e-8049-d8ac15669541 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:29.125549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57accf-a896-490e-8049-d8ac15669541 None admin] PATH_INFO: `/v3/services/network` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:29.126327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57accf-a896-490e-8049-d8ac15669541 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:29.127950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57accf-a896-490e-8049-d8ac15669541 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-bc57accf-a896-490e-8049-d8ac15669541 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:29.142556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:29.143767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 09:41:29.146030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:29.148543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 395/776] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:28 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:29.156286 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed6290-81b5-42f8-9d1a-6648b8d901fa None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:29.312199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed6290-81b5-42f8-9d1a-6648b8d901fa None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:29.357260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d85258-8018-4a32-b146-9d9872153f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:29.357875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d85258-8018-4a32-b146-9d9872153f6d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:29.358160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d85258-8018-4a32-b146-9d9872153f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:29.358433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d85258-8018-4a32-b146-9d9872153f6d None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:29.359072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d85258-8018-4a32-b146-9d9872153f6d None admin] RBAC: Adding query filter params (name=network) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:29.359440 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d85258-8018-4a32-b146-9d9872153f6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:29.360819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d85258-8018-4a32-b146-9d9872153f6d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:29.362090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:29.369350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 382/777] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:29.374618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57accf-a896-490e-8049-d8ac15669541 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:29.388116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c084207-11ff-4552-8446-06052bbcf351 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:29.388406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c084207-11ff-4552-8446-06052bbcf351 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:29.388686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c084207-11ff-4552-8446-06052bbcf351 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:29.393380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 383/778] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:29.399090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b717060-89aa-4f05-a477-ce1c53cb56da None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:29.399374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b717060-89aa-4f05-a477-ce1c53cb56da None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:29.399718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b717060-89aa-4f05-a477-ce1c53cb56da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:29.430045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9b717060-89aa-4f05-a477-ce1c53cb56da None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:29.431402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9b717060-89aa-4f05-a477-ce1c53cb56da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:29.457738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:29.547550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57accf-a896-490e-8049-d8ac15669541 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:29.571654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 384/779] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:29.604592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0443-bb0a-436c-a112-c0a00538f568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:29.605414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0443-bb0a-436c-a112-c0a00538f568 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:29.605623 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b717060-89aa-4f05-a477-ce1c53cb56da None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:29.605901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0443-bb0a-436c-a112-c0a00538f568 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:29.606280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0443-bb0a-436c-a112-c0a00538f568 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:29.607122 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0443-bb0a-436c-a112-c0a00538f568 None admin] RBAC: Adding query filter params (type=network) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:29.607544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0443-bb0a-436c-a112-c0a00538f568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:29.608994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0443-bb0a-436c-a112-c0a00538f568 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:29.618756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 396/780] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:29.620817 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0443-bb0a-436c-a112-c0a00538f568 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:29.788804 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b717060-89aa-4f05-a477-ce1c53cb56da None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:29.806652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0443-bb0a-436c-a112-c0a00538f568 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:29.829093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:29.839980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:29.848805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e46227-9d67-4277-bfc0-834082a4aa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:29.849470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e46227-9d67-4277-bfc0-834082a4aa85 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:29.849750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e46227-9d67-4277-bfc0-834082a4aa85 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:29.850024 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e46227-9d67-4277-bfc0-834082a4aa85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:29.858650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1dbdf4-d501-482c-9709-1fed5c213f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:29.859266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1dbdf4-d501-482c-9709-1fed5c213f66 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:29.859545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1dbdf4-d501-482c-9709-1fed5c213f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:29.859819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1dbdf4-d501-482c-9709-1fed5c213f66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:29.860526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1dbdf4-d501-482c-9709-1fed5c213f66 None admin] RBAC: Adding query filter params (interface=public, service_id=c126049bb7df4f58aa7c8bd4b06c41ac, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:29.860836 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1dbdf4-d501-482c-9709-1fed5c213f66 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:29.861309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e46227-9d67-4277-bfc0-834082a4aa85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:29.862217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1dbdf4-d501-482c-9709-1fed5c213f66 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-d6e46227-9d67-4277-bfc0-834082a4aa85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:29.867020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:29.867934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:29.868737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 385/782] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:29.877049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 397/782] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Aug 11 09:41:29 2022] GET /identity/v3/endpoints?service_id=c126049bb7df4f58aa7c8bd4b06c41ac&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:29.877209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e46227-9d67-4277-bfc0-834082a4aa85 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:29.877288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1dbdf4-d501-482c-9709-1fed5c213f66 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:30.031684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e46227-9d67-4277-bfc0-834082a4aa85 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:30.070660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1dbdf4-d501-482c-9709-1fed5c213f66 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:30.084741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63c96d-059c-425a-b51f-1e9817dab29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:30.085401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c96d-059c-425a-b51f-1e9817dab29c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:30.085684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c96d-059c-425a-b51f-1e9817dab29c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:30.085959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c96d-059c-425a-b51f-1e9817dab29c None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:30.086575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63c96d-059c-425a-b51f-1e9817dab29c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:30.087976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63c96d-059c-425a-b51f-1e9817dab29c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:30.101192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 386/783] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Aug 11 09:41:29 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:30.133959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592dc37-d94e-4e96-a450-4a3065401e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:30.134586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592dc37-d94e-4e96-a450-4a3065401e66 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:30.134874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592dc37-d94e-4e96-a450-4a3065401e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:30.135157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592dc37-d94e-4e96-a450-4a3065401e66 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:30.135845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592dc37-d94e-4e96-a450-4a3065401e66 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:30.136156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592dc37-d94e-4e96-a450-4a3065401e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:30.137558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592dc37-d94e-4e96-a450-4a3065401e66 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:30.147306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 398/784] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:30.152882 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63c96d-059c-425a-b51f-1e9817dab29c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:30.233376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:30.304078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63c96d-059c-425a-b51f-1e9817dab29c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:30.344222 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ed32b-ed7b-461e-956f-6c47280554bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:30.344774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ed32b-ed7b-461e-956f-6c47280554bc None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:30.345077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ed32b-ed7b-461e-956f-6c47280554bc None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:30.345354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ed32b-ed7b-461e-956f-6c47280554bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:30.356336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ed32b-ed7b-461e-956f-6c47280554bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:30.357840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ed32b-ed7b-461e-956f-6c47280554bc None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:30.366400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57426d87-5385-4c13-8bb1-9c4eeccaacaf None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:30.366704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57426d87-5385-4c13-8bb1-9c4eeccaacaf None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:30.366955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57426d87-5385-4c13-8bb1-9c4eeccaacaf None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:30.368636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 399/785] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:30.374627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c574b-041f-4393-ae0f-f4d3ef0b4bd4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:30.382879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c574b-041f-4393-ae0f-f4d3ef0b4bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:30.382879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c574b-041f-4393-ae0f-f4d3ef0b4bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-f69ed32b-ed7b-461e-956f-6c47280554bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:30.398632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:30.399609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:30.400457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:30.400457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:30.400457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:30.400457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 387/786] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:30.403913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-999c574b-041f-4393-ae0f-f4d3ef0b4bd4 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:30.405547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ed32b-ed7b-461e-956f-6c47280554bc None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:30.406921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-999c574b-041f-4393-ae0f-f4d3ef0b4bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:30.443343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:30.486120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 400/787] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:30.509802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c574b-041f-4393-ae0f-f4d3ef0b4bd4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:30.534622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ed32b-ed7b-461e-956f-6c47280554bc None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:30.545389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:30.598300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851505b2-894e-4978-8012-10c8efc10bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:30.599737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851505b2-894e-4978-8012-10c8efc10bda None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:30.600067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851505b2-894e-4978-8012-10c8efc10bda None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:30.600341 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851505b2-894e-4978-8012-10c8efc10bda None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:30.600988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851505b2-894e-4978-8012-10c8efc10bda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:30.601309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851505b2-894e-4978-8012-10c8efc10bda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:30.602668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851505b2-894e-4978-8012-10c8efc10bda None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:30.624154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 388/788] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:30.630042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851505b2-894e-4978-8012-10c8efc10bda None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:30.680767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c574b-041f-4393-ae0f-f4d3ef0b4bd4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:30.720168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a7117-d7d2-413a-854c-92deb687689e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:30.720802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a7117-d7d2-413a-854c-92deb687689e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:30.721140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a7117-d7d2-413a-854c-92deb687689e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:30.721558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a7117-d7d2-413a-854c-92deb687689e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:30.733261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a7117-d7d2-413a-854c-92deb687689e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-754a7117-d7d2-413a-854c-92deb687689e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:30.738570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:30.739478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:30.740280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 401/789] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:30.747328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a7117-d7d2-413a-854c-92deb687689e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:30.777456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:30.888687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851505b2-894e-4978-8012-10c8efc10bda None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:30.935167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a7117-d7d2-413a-854c-92deb687689e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:30.936477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268f00b-c719-41fe-a8c6-1e20e1a48ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:30.937450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268f00b-c719-41fe-a8c6-1e20e1a48ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:30.939320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268f00b-c719-41fe-a8c6-1e20e1a48ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:30.939796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268f00b-c719-41fe-a8c6-1e20e1a48ce5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:30.957943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268f00b-c719-41fe-a8c6-1e20e1a48ce5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:30.961399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268f00b-c719-41fe-a8c6-1e20e1a48ce5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-0268f00b-c719-41fe-a8c6-1e20e1a48ce5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:30.989147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:30.990119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:30.991025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:30.991025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:30.991025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:30.991025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:30.991025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:30.991025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:30.991025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:30.991025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:30.991025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:30.991025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:30.996380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 389/790] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Aug 11 09:41:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:31.000683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268f00b-c719-41fe-a8c6-1e20e1a48ce5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:31.018306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd7873-eb6d-41df-971d-98025ad301c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:31.018987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd7873-eb6d-41df-971d-98025ad301c3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.019335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd7873-eb6d-41df-971d-98025ad301c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.019613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd7873-eb6d-41df-971d-98025ad301c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.020293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd7873-eb6d-41df-971d-98025ad301c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:31.020598 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd7873-eb6d-41df-971d-98025ad301c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:31.022020 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd7873-eb6d-41df-971d-98025ad301c3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:31.033734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:31.037478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 402/791] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:31.046207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd7873-eb6d-41df-971d-98025ad301c3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:31.131249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268f00b-c719-41fe-a8c6-1e20e1a48ce5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:31.179898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6143d5-5b5f-400e-9b52-fad6d207e72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:31.180431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6143d5-5b5f-400e-9b52-fad6d207e72a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.184264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6143d5-5b5f-400e-9b52-fad6d207e72a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.184264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6143d5-5b5f-400e-9b52-fad6d207e72a None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.184264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6143d5-5b5f-400e-9b52-fad6d207e72a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:31.184264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6143d5-5b5f-400e-9b52-fad6d207e72a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:31.186997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6143d5-5b5f-400e-9b52-fad6d207e72a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:31.220663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 390/792] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Aug 11 09:41:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:31.226074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6143d5-5b5f-400e-9b52-fad6d207e72a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:31.236282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:31.247562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd7873-eb6d-41df-971d-98025ad301c3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:31.297727 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881545f-aeac-425b-a1d2-94bd51847c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:31.298619 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881545f-aeac-425b-a1d2-94bd51847c3c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.299085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881545f-aeac-425b-a1d2-94bd51847c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.299508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881545f-aeac-425b-a1d2-94bd51847c3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.319749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881545f-aeac-425b-a1d2-94bd51847c3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:31.321507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881545f-aeac-425b-a1d2-94bd51847c3c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:31.333279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a881545f-aeac-425b-a1d2-94bd51847c3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:31.349434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:31.350425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:31.351332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:31.351332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:31.351332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:31.351332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 403/793] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:31.354826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881545f-aeac-425b-a1d2-94bd51847c3c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:31.364081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6143d5-5b5f-400e-9b52-fad6d207e72a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:31.411414 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c7f3a-d970-4d6d-a718-51b0b62de832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:31.412511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c7f3a-d970-4d6d-a718-51b0b62de832 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.412972 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c7f3a-d970-4d6d-a718-51b0b62de832 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.413447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c7f3a-d970-4d6d-a718-51b0b62de832 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.427274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c7f3a-d970-4d6d-a718-51b0b62de832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:31.428996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c7f3a-d970-4d6d-a718-51b0b62de832 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-860c7f3a-d970-4d6d-a718-51b0b62de832 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:31.446051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:31.447084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:31.447970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:31.447970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:31.447970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:31.450228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 391/794] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:31.457668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c7f3a-d970-4d6d-a718-51b0b62de832 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:31.517835 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881545f-aeac-425b-a1d2-94bd51847c3c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:31.561322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf75ae-8f9c-4b52-855b-b8f2735caf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:31.561950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf75ae-8f9c-4b52-855b-b8f2735caf9f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.562227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf75ae-8f9c-4b52-855b-b8f2735caf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.562496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf75ae-8f9c-4b52-855b-b8f2735caf9f None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.563156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf75ae-8f9c-4b52-855b-b8f2735caf9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:31.563484 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf75ae-8f9c-4b52-855b-b8f2735caf9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:31.564876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf75ae-8f9c-4b52-855b-b8f2735caf9f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:31.571602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c7f3a-d970-4d6d-a718-51b0b62de832 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:31.578688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 404/795] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:31.579921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65cf4-010a-4b27-9065-c4fb348524bd None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.580202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65cf4-010a-4b27-9065-c4fb348524bd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.580470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec65cf4-010a-4b27-9065-c4fb348524bd None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.585056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 405/796] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:31.585056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d9215-0c9d-4a77-a713-029e8e296f14 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.585056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d9215-0c9d-4a77-a713-029e8e296f14 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.585056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d9215-0c9d-4a77-a713-029e8e296f14 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.585397 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 406/797] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:31.585989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d9215-0c9d-4a77-a713-029e8e296f14 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:31.588940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:31.617130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffffb5-2f00-424d-9fd6-1dbfc9b00eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:31.617811 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffffb5-2f00-424d-9fd6-1dbfc9b00eeb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.618019 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffffb5-2f00-424d-9fd6-1dbfc9b00eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.618435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffffb5-2f00-424d-9fd6-1dbfc9b00eeb None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.619078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffffb5-2f00-424d-9fd6-1dbfc9b00eeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:31.619671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffffb5-2f00-424d-9fd6-1dbfc9b00eeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:31.621319 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffffb5-2f00-424d-9fd6-1dbfc9b00eeb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:31.622939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:31.635901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 392/798] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:31.637287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb63152-3325-47b2-853d-6d445196c626 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.637641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb63152-3325-47b2-853d-6d445196c626 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.637985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb63152-3325-47b2-853d-6d445196c626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.656371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ffb63152-3325-47b2-853d-6d445196c626 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:31.657626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ffb63152-3325-47b2-853d-6d445196c626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:31.700758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d9215-0c9d-4a77-a713-029e8e296f14 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:31.736803 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836152-3aea-44be-9e6f-5ebce617b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:31.737465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836152-3aea-44be-9e6f-5ebce617b953 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.737774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836152-3aea-44be-9e6f-5ebce617b953 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.738102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e836152-3aea-44be-9e6f-5ebce617b953 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.741335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 393/799] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:31.742628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be825ec-bda1-4a5d-b686-895f07b8b1de None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.742914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be825ec-bda1-4a5d-b686-895f07b8b1de None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.743185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be825ec-bda1-4a5d-b686-895f07b8b1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.753150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e836152-3aea-44be-9e6f-5ebce617b953 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:31.753150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e836152-3aea-44be-9e6f-5ebce617b953 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0e836152-3aea-44be-9e6f-5ebce617b953 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:31.765463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:31.766445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:31.767451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:31.767451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:31.767451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:31.767451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:31.767451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:31.767451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:31.767451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:31.767451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:31.767451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:41:31.767451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:31.767451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 407/800] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Aug 11 09:41:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:31.770512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836152-3aea-44be-9e6f-5ebce617b953 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:31.773470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6be825ec-bda1-4a5d-b686-895f07b8b1de None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:31.775498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6be825ec-bda1-4a5d-b686-895f07b8b1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:31.836874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:31.865044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 394/801] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:31.865044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be825ec-bda1-4a5d-b686-895f07b8b1de None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:31.889761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e836152-3aea-44be-9e6f-5ebce617b953 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:31.922828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:31.929276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2a8f6-5671-4d4a-a6a0-553f5b388a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:31.929944 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a8f6-5671-4d4a-a6a0-553f5b388a57 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:31.930227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a8f6-5671-4d4a-a6a0-553f5b388a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:31.930708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2a8f6-5671-4d4a-a6a0-553f5b388a57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:31.937789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2a8f6-5671-4d4a-a6a0-553f5b388a57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:31.939394 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e2a8f6-5671-4d4a-a6a0-553f5b388a57 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-86e2a8f6-5671-4d4a-a6a0-553f5b388a57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:31.951866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:31.952743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:31.953961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:31.953961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:31.953961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:31.953961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:31.953961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:31.953961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:31.953961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:31.953961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:31.953961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 408/802] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:31.953961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2a8f6-5671-4d4a-a6a0-553f5b388a57 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:31.976209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be825ec-bda1-4a5d-b686-895f07b8b1de None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:32.010877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f028d41-0617-487e-ada3-e69f253dd4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:32.011510 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028d41-0617-487e-ada3-e69f253dd4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.011802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028d41-0617-487e-ada3-e69f253dd4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.012074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f028d41-0617-487e-ada3-e69f253dd4a5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.012657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f028d41-0617-487e-ada3-e69f253dd4a5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:32.014061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f028d41-0617-487e-ada3-e69f253dd4a5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-8f028d41-0617-487e-ada3-e69f253dd4a5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:32.023225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:32.024212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:32.029846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 395/803] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Aug 11 09:41:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:32.030655 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f028d41-0617-487e-ada3-e69f253dd4a5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:32.073474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e2a8f6-5671-4d4a-a6a0-553f5b388a57 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:32.102428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:32.117415 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223bad5-fe37-4d30-9128-4fa10b2476c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:32.118035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223bad5-fe37-4d30-9128-4fa10b2476c0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.118316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223bad5-fe37-4d30-9128-4fa10b2476c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.118688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223bad5-fe37-4d30-9128-4fa10b2476c0 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.119377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223bad5-fe37-4d30-9128-4fa10b2476c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:32.120236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223bad5-fe37-4d30-9128-4fa10b2476c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:32.122011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223bad5-fe37-4d30-9128-4fa10b2476c0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:32.131778 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f028d41-0617-487e-ada3-e69f253dd4a5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:32.148214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 409/804] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Aug 11 09:41:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:32.148214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223bad5-fe37-4d30-9128-4fa10b2476c0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:32.153770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:32.172067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2bf73-306c-4958-880e-9e865163f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:32.172707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2bf73-306c-4958-880e-9e865163f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.172995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2bf73-306c-4958-880e-9e865163f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.173292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2bf73-306c-4958-880e-9e865163f5d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.185207 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2bf73-306c-4958-880e-9e865163f5d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-78a2bf73-306c-4958-880e-9e865163f5d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:32.191541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:32.192733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:32.193592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:32.197152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 396/805] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:32.198007 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2bf73-306c-4958-880e-9e865163f5d6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:32.260406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223bad5-fe37-4d30-9128-4fa10b2476c0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:32.295832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112c3a2-7e08-449f-9b2b-83b455738602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:32.296762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112c3a2-7e08-449f-9b2b-83b455738602 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.297214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112c3a2-7e08-449f-9b2b-83b455738602 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.297502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112c3a2-7e08-449f-9b2b-83b455738602 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.298197 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112c3a2-7e08-449f-9b2b-83b455738602 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:32.298510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112c3a2-7e08-449f-9b2b-83b455738602 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:32.299924 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112c3a2-7e08-449f-9b2b-83b455738602 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:32.303055 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2bf73-306c-4958-880e-9e865163f5d6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:32.311642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 410/806] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:32.313896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112c3a2-7e08-449f-9b2b-83b455738602 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:32.341533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d45a5b-385d-4065-ad29-c4a299970c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:32.342157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45a5b-385d-4065-ad29-c4a299970c6a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.342430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45a5b-385d-4065-ad29-c4a299970c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.342708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45a5b-385d-4065-ad29-c4a299970c6a None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.343346 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d45a5b-385d-4065-ad29-c4a299970c6a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:32.343653 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d45a5b-385d-4065-ad29-c4a299970c6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:32.345039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d45a5b-385d-4065-ad29-c4a299970c6a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:32.350187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 397/807] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Aug 11 09:41:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:32.350970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d45a5b-385d-4065-ad29-c4a299970c6a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:32.368753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:32.398212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:32.432493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112c3a2-7e08-449f-9b2b-83b455738602 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:32.459249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d45a5b-385d-4065-ad29-c4a299970c6a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:32.469449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30db7e7-e1fe-4ddd-9a9f-65e47871827a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:32.470071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db7e7-e1fe-4ddd-9a9f-65e47871827a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.470351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db7e7-e1fe-4ddd-9a9f-65e47871827a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.470623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30db7e7-e1fe-4ddd-9a9f-65e47871827a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.494048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78064f-b47f-4732-80c1-0e86f4b2da69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:32.494655 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78064f-b47f-4732-80c1-0e86f4b2da69 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.494929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78064f-b47f-4732-80c1-0e86f4b2da69 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.495194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78064f-b47f-4732-80c1-0e86f4b2da69 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.495848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78064f-b47f-4732-80c1-0e86f4b2da69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:32.496143 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78064f-b47f-4732-80c1-0e86f4b2da69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:32.497525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78064f-b47f-4732-80c1-0e86f4b2da69 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:32.505562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 398/808] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:32.506490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78064f-b47f-4732-80c1-0e86f4b2da69 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:32.541972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30db7e7-e1fe-4ddd-9a9f-65e47871827a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:32.543645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30db7e7-e1fe-4ddd-9a9f-65e47871827a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:32.582897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:32.606464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78064f-b47f-4732-80c1-0e86f4b2da69 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:32.608707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:32.639644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19bb3b-3a42-4998-9692-e0dc6d94b2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:32.640292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19bb3b-3a42-4998-9692-e0dc6d94b2be None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.640573 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19bb3b-3a42-4998-9692-e0dc6d94b2be None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.640847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19bb3b-3a42-4998-9692-e0dc6d94b2be None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/813d8cff9d7d49d5995bfc272fee88f3/roles/c6c1ea7ecf3d4e00a138f97d91b14f73` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.641490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 411/809] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:41:32.642320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30db7e7-e1fe-4ddd-9a9f-65e47871827a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:32.656205 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19bb3b-3a42-4998-9692-e0dc6d94b2be None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=813d8cff9d7d49d5995bfc272fee88f3, role_id=c6c1ea7ecf3d4e00a138f97d91b14f73)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:32.657672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19bb3b-3a42-4998-9692-e0dc6d94b2be None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:32.713781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 399/810] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:41:32 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/813d8cff9d7d49d5995bfc272fee88f3/roles/c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:41:32.715079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fedf2-dc65-4835-b435-cb0bacbecd61 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.715365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fedf2-dc65-4835-b435-cb0bacbecd61 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.715668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fedf2-dc65-4835-b435-cb0bacbecd61 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.716960 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 400/811] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:32.717745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fedf2-dc65-4835-b435-cb0bacbecd61 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:32.757242 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30db7e7-e1fe-4ddd-9a9f-65e47871827a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:32.797334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f58274-3bd7-41f9-a90d-8d1150f55676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:32.798234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f58274-3bd7-41f9-a90d-8d1150f55676 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.798516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f58274-3bd7-41f9-a90d-8d1150f55676 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.798787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f58274-3bd7-41f9-a90d-8d1150f55676 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.799444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f58274-3bd7-41f9-a90d-8d1150f55676 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:32.799746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f58274-3bd7-41f9-a90d-8d1150f55676 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:32.801442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f58274-3bd7-41f9-a90d-8d1150f55676 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:32.811753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 412/812] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Aug 11 09:41:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:32.811753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f58274-3bd7-41f9-a90d-8d1150f55676 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:32.813388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:32.845330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fedf2-dc65-4835-b435-cb0bacbecd61 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:32.885768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc80a1-9d6d-473a-b964-94cafd5ae729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:32.886388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc80a1-9d6d-473a-b964-94cafd5ae729 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.886662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc80a1-9d6d-473a-b964-94cafd5ae729 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.886930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc80a1-9d6d-473a-b964-94cafd5ae729 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.898201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc80a1-9d6d-473a-b964-94cafd5ae729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:32.899655 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc80a1-9d6d-473a-b964-94cafd5ae729 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-81fc80a1-9d6d-473a-b964-94cafd5ae729 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:32.915157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:32.916105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:32.917815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:32.917815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:32.917815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:32.917815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 401/813] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:32.918033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b8143-2183-4610-83c8-76983ca34772 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.918274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b8143-2183-4610-83c8-76983ca34772 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.918541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b8143-2183-4610-83c8-76983ca34772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.927652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f58274-3bd7-41f9-a90d-8d1150f55676 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:32.931396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:32.936928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-aa4b8143-2183-4610-83c8-76983ca34772 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:32.938324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-aa4b8143-2183-4610-83c8-76983ca34772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:32.964993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f2bb2-d770-45f4-a8d5-7734db90658a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:32.965743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f2bb2-d770-45f4-a8d5-7734db90658a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:32.965989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f2bb2-d770-45f4-a8d5-7734db90658a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:32.966206 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f2bb2-d770-45f4-a8d5-7734db90658a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:32.977816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f2bb2-d770-45f4-a8d5-7734db90658a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:32.979252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f2bb2-d770-45f4-a8d5-7734db90658a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5e3f2bb2-d770-45f4-a8d5-7734db90658a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:32.994232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:32.995194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:32.996080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:32.996080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:32.996080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:32.996080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 413/814] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:32.996080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f2bb2-d770-45f4-a8d5-7734db90658a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:33.020644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 402/815] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:33.021904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b8143-2183-4610-83c8-76983ca34772 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:33.069131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:33.114891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f2bb2-d770-45f4-a8d5-7734db90658a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:33.132387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b8143-2183-4610-83c8-76983ca34772 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:33.152306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0ff37-7369-474d-afb2-ed3e8c9a27ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:33.152917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0ff37-7369-474d-afb2-ed3e8c9a27ce None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:33.153234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0ff37-7369-474d-afb2-ed3e8c9a27ce None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:33.153507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0ff37-7369-474d-afb2-ed3e8c9a27ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:33.154171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0ff37-7369-474d-afb2-ed3e8c9a27ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:33.155957 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0ff37-7369-474d-afb2-ed3e8c9a27ce None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:33.170203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff7847-5d88-4447-8931-40a364558680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:33.170813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff7847-5d88-4447-8931-40a364558680 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:33.171094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff7847-5d88-4447-8931-40a364558680 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:33.171367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff7847-5d88-4447-8931-40a364558680 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:33.172041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff7847-5d88-4447-8931-40a364558680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:33.172347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff7847-5d88-4447-8931-40a364558680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:33.173750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff7847-5d88-4447-8931-40a364558680 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:33.176197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:33.186086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 403/816] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:33.186926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff7847-5d88-4447-8931-40a364558680 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:33.263572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 414/817] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:41:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:33.274120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0ff37-7369-474d-afb2-ed3e8c9a27ce None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:33.292471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff7847-5d88-4447-8931-40a364558680 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:33.346988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae7b30-48ee-4c10-816c-ab081803e3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:33.347738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae7b30-48ee-4c10-816c-ab081803e3fa None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:33.348124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae7b30-48ee-4c10-816c-ab081803e3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:33.348499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae7b30-48ee-4c10-816c-ab081803e3fa None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:33.349334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae7b30-48ee-4c10-816c-ab081803e3fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:33.349763 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae7b30-48ee-4c10-816c-ab081803e3fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:33.351267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae7b30-48ee-4c10-816c-ab081803e3fa None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:33.372895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 404/818] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:33.374174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae7b30-48ee-4c10-816c-ab081803e3fa None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:33.431938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:33.436088 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0ff37-7369-474d-afb2-ed3e8c9a27ce None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:33.452048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:33.484090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29326de0-e2c1-410c-b4c6-351ee33508b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:33.484745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29326de0-e2c1-410c-b4c6-351ee33508b6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:33.485801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29326de0-e2c1-410c-b4c6-351ee33508b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:33.485801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29326de0-e2c1-410c-b4c6-351ee33508b6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:33.485999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29326de0-e2c1-410c-b4c6-351ee33508b6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:33.487289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29326de0-e2c1-410c-b4c6-351ee33508b6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:33.491882 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae7b30-48ee-4c10-816c-ab081803e3fa None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-29326de0-e2c1-410c-b4c6-351ee33508b6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:33.500511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:33.503351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:33.505487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 415/819] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:33 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:33.506291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29326de0-e2c1-410c-b4c6-351ee33508b6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:33.537424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2866f-7e5c-4338-ae9b-3c123fe0d7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:33.538690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2866f-7e5c-4338-ae9b-3c123fe0d7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:33.539332 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2866f-7e5c-4338-ae9b-3c123fe0d7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:33.539704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2866f-7e5c-4338-ae9b-3c123fe0d7f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:33.547800 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2866f-7e5c-4338-ae9b-3c123fe0d7f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:33.549357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2866f-7e5c-4338-ae9b-3c123fe0d7f6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-78f2866f-7e5c-4338-ae9b-3c123fe0d7f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:33.562540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:33.563581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:33.564529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:33.564529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:33.564529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:33.564529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:33.564529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:33.564529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:33.564529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:33.564529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:33.564529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:33.564529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:33.564529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 405/820] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:33.565139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2866f-7e5c-4338-ae9b-3c123fe0d7f6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:33.631665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29326de0-e2c1-410c-b4c6-351ee33508b6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:33.670955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29986fd-0d0f-4a8f-98b7-83fdf25bb10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:33.671778 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29986fd-0d0f-4a8f-98b7-83fdf25bb10e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:33.672157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29986fd-0d0f-4a8f-98b7-83fdf25bb10e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:33.672529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29986fd-0d0f-4a8f-98b7-83fdf25bb10e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:33.673653 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2866f-7e5c-4338-ae9b-3c123fe0d7f6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:33.680784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29986fd-0d0f-4a8f-98b7-83fdf25bb10e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:33.682491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29986fd-0d0f-4a8f-98b7-83fdf25bb10e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:33.687640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e29986fd-0d0f-4a8f-98b7-83fdf25bb10e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:33.696499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:33.702021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:33.702954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:33.702954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:33.702954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:33.702954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:33.702954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:33.702954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:33.702954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:33.702954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:33.702954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 416/821] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:33.702954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29986fd-0d0f-4a8f-98b7-83fdf25bb10e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:33.711493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef79625-44ff-40c0-9bac-7758d6b0f6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:33.712166 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef79625-44ff-40c0-9bac-7758d6b0f6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:33.712455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef79625-44ff-40c0-9bac-7758d6b0f6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:33.712732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef79625-44ff-40c0-9bac-7758d6b0f6a2 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:33.713420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef79625-44ff-40c0-9bac-7758d6b0f6a2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:33.713726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef79625-44ff-40c0-9bac-7758d6b0f6a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:33.715256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef79625-44ff-40c0-9bac-7758d6b0f6a2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:33.720578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 406/822] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:33.721551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef79625-44ff-40c0-9bac-7758d6b0f6a2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:33.744781 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:33.819829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29986fd-0d0f-4a8f-98b7-83fdf25bb10e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:33.828378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef79625-44ff-40c0-9bac-7758d6b0f6a2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:33.865697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306c868-a285-474f-b1bb-2971bb542e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:33.866436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306c868-a285-474f-b1bb-2971bb542e41 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:33.866783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306c868-a285-474f-b1bb-2971bb542e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:33.867126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306c868-a285-474f-b1bb-2971bb542e41 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:33.867963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306c868-a285-474f-b1bb-2971bb542e41 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:33.868330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306c868-a285-474f-b1bb-2971bb542e41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:33.869581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1d080-ed85-4f75-8961-3a158cfebf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:33.869951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306c868-a285-474f-b1bb-2971bb542e41 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:33.870575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d080-ed85-4f75-8961-3a158cfebf92 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:33.870999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d080-ed85-4f75-8961-3a158cfebf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:33.871418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d080-ed85-4f75-8961-3a158cfebf92 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:33.872255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1d080-ed85-4f75-8961-3a158cfebf92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:33.872721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1d080-ed85-4f75-8961-3a158cfebf92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:33.874369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1d080-ed85-4f75-8961-3a158cfebf92 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:33.887929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 407/823] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:33.889150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1d080-ed85-4f75-8961-3a158cfebf92 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:33.896110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 417/824] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Aug 11 09:41:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:33.897227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306c868-a285-474f-b1bb-2971bb542e41 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:33.930748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:34.002254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:34.003415 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1d080-ed85-4f75-8961-3a158cfebf92 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:34.009555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306c868-a285-474f-b1bb-2971bb542e41 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:34.042208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92038e7-8eff-41d8-a3f3-ddf1b93fdbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:34.043170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92038e7-8eff-41d8-a3f3-ddf1b93fdbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:34.043680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92038e7-8eff-41d8-a3f3-ddf1b93fdbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:34.044187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92038e7-8eff-41d8-a3f3-ddf1b93fdbc9 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:34.045002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92038e7-8eff-41d8-a3f3-ddf1b93fdbc9 None admin] RBAC: Adding query filter params (type=network) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:34.045565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92038e7-8eff-41d8-a3f3-ddf1b93fdbc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:34.046126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed234f14-6c8f-46b3-87f6-823167fdb23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:34.046800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed234f14-6c8f-46b3-87f6-823167fdb23e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:34.047147 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed234f14-6c8f-46b3-87f6-823167fdb23e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:34.047486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed234f14-6c8f-46b3-87f6-823167fdb23e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:34.047762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92038e7-8eff-41d8-a3f3-ddf1b93fdbc9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:34.048340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed234f14-6c8f-46b3-87f6-823167fdb23e None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=34fa13a87ebe4b789ca1884e5aeb9237) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:34.048714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed234f14-6c8f-46b3-87f6-823167fdb23e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:34.050266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed234f14-6c8f-46b3-87f6-823167fdb23e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:34.055811 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 408/825] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:34.056954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92038e7-8eff-41d8-a3f3-ddf1b93fdbc9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:34.057200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 418/826] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:33 2022] GET /identity/v3/role_assignments?user.id=34fa13a87ebe4b789ca1884e5aeb9237&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:34.060838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed234f14-6c8f-46b3-87f6-823167fdb23e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:34.167050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:34.170697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92038e7-8eff-41d8-a3f3-ddf1b93fdbc9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:34.170916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed234f14-6c8f-46b3-87f6-823167fdb23e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:34.210464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f299d1d-690a-4662-9957-0d732474eeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:34.211074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93474b4-5dc2-4d55-a6fc-f2b183423110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:34.211198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f299d1d-690a-4662-9957-0d732474eeb4 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:34.211359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f299d1d-690a-4662-9957-0d732474eeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:34.211633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f299d1d-690a-4662-9957-0d732474eeb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:34.212304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f299d1d-690a-4662-9957-0d732474eeb4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:34.212603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93474b4-5dc2-4d55-a6fc-f2b183423110 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:34.213084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93474b4-5dc2-4d55-a6fc-f2b183423110 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:34.213575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93474b4-5dc2-4d55-a6fc-f2b183423110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:34.213701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f299d1d-690a-4662-9957-0d732474eeb4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:34.225766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93474b4-5dc2-4d55-a6fc-f2b183423110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:34.227272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93474b4-5dc2-4d55-a6fc-f2b183423110 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b93474b4-5dc2-4d55-a6fc-f2b183423110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:34.243933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:34.245258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:34.246134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:34.246134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:34.246134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:34.246134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 409/827] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:34.254771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93474b4-5dc2-4d55-a6fc-f2b183423110 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:34.304176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 419/828] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:41:34 2022] POST /identity/v3/endpoints => generated 337 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:34.376073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93474b4-5dc2-4d55-a6fc-f2b183423110 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:34.424722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4829f-6b95-40ba-b2cf-45c71a82da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:34.426210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4829f-6b95-40ba-b2cf-45c71a82da05 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:34.426645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4829f-6b95-40ba-b2cf-45c71a82da05 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:34.427214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4829f-6b95-40ba-b2cf-45c71a82da05 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:34.428120 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4829f-6b95-40ba-b2cf-45c71a82da05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:34.428592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4829f-6b95-40ba-b2cf-45c71a82da05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:34.430166 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4829f-6b95-40ba-b2cf-45c71a82da05 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:34.449976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 410/829] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:34.450354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f299d1d-690a-4662-9957-0d732474eeb4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:34.477401 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:34.578282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f299d1d-690a-4662-9957-0d732474eeb4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:34.618331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8516de-aa64-48a7-9c58-5add9a776be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:34.618983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8516de-aa64-48a7-9c58-5add9a776be2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:34.619265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8516de-aa64-48a7-9c58-5add9a776be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:34.619541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8516de-aa64-48a7-9c58-5add9a776be2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:34.627309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8516de-aa64-48a7-9c58-5add9a776be2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:34.629353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8516de-aa64-48a7-9c58-5add9a776be2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2a8516de-aa64-48a7-9c58-5add9a776be2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:34.642274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:34.643452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:34.644982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:34.644982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:34.644982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:34.644982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:34.644982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:34.644982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:34.644982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:34.644982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:34.647409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 420/830] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:34.659061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4829f-6b95-40ba-b2cf-45c71a82da05 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:34.670715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:34.779046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4829f-6b95-40ba-b2cf-45c71a82da05 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:34.813466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b66fe-1c09-49f2-81cd-b8b5d5c1571b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:34.814071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b66fe-1c09-49f2-81cd-b8b5d5c1571b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:34.814345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b66fe-1c09-49f2-81cd-b8b5d5c1571b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:34.814613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b66fe-1c09-49f2-81cd-b8b5d5c1571b None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:34.815307 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b66fe-1c09-49f2-81cd-b8b5d5c1571b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:34.815606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b66fe-1c09-49f2-81cd-b8b5d5c1571b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:34.817065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b66fe-1c09-49f2-81cd-b8b5d5c1571b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:34.829308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 411/831] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:34.835878 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8516de-aa64-48a7-9c58-5add9a776be2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:34.913675 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f9f88-55ae-4731-9c4c-f6e9bfb0f135 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:34.914331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f9f88-55ae-4731-9c4c-f6e9bfb0f135 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:34.914838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f9f88-55ae-4731-9c4c-f6e9bfb0f135 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:34.915639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:34.917670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 412/832] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:34.924242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8526b6-58fc-4a0e-886b-1c9a02e4c734 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:34.924530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8526b6-58fc-4a0e-886b-1c9a02e4c734 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:34.924846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8526b6-58fc-4a0e-886b-1c9a02e4c734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:34.943924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6e8526b6-58fc-4a0e-886b-1c9a02e4c734 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:34.945174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6e8526b6-58fc-4a0e-886b-1c9a02e4c734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:34.946356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8516de-aa64-48a7-9c58-5add9a776be2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:34.984068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee93e0f-b6a8-44b7-9a95-5c379864d4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:34.985172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee93e0f-b6a8-44b7-9a95-5c379864d4c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:34.985659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee93e0f-b6a8-44b7-9a95-5c379864d4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:34.986134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee93e0f-b6a8-44b7-9a95-5c379864d4c6 None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/915ee44218064af896020d008ff31957/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:34.988511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee93e0f-b6a8-44b7-9a95-5c379864d4c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=915ee44218064af896020d008ff31957, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:34.990264 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee93e0f-b6a8-44b7-9a95-5c379864d4c6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:35.010403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:35.020526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 413/833] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:35.040061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8526b6-58fc-4a0e-886b-1c9a02e4c734 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:35.047992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 421/834] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:41:34 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/915ee44218064af896020d008ff31957/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:41:35.158847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8526b6-58fc-4a0e-886b-1c9a02e4c734 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:35.201829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239745bf-b2d9-4919-93f7-307c1aa60eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:35.202547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239745bf-b2d9-4919-93f7-307c1aa60eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:35.202831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239745bf-b2d9-4919-93f7-307c1aa60eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:35.203108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239745bf-b2d9-4919-93f7-307c1aa60eb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:35.215594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239745bf-b2d9-4919-93f7-307c1aa60eb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-239745bf-b2d9-4919-93f7-307c1aa60eb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:35.220577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:35.221515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:41:35.222876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:35.227630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 414/835] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:41:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:35.228832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee93e0f-b6a8-44b7-9a95-5c379864d4c6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:35.337269 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee93e0f-b6a8-44b7-9a95-5c379864d4c6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:35.376650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dcc36-308b-45f5-a13f-f24f04dbce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:35.377288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dcc36-308b-45f5-a13f-f24f04dbce13 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:35.377570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dcc36-308b-45f5-a13f-f24f04dbce13 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:35.377855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dcc36-308b-45f5-a13f-f24f04dbce13 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:35.388178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dcc36-308b-45f5-a13f-f24f04dbce13 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:35.388178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dcc36-308b-45f5-a13f-f24f04dbce13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:35.388178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dcc36-308b-45f5-a13f-f24f04dbce13 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:35.388178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:35.393142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 422/836] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:41:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:35.395180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239745bf-b2d9-4919-93f7-307c1aa60eb2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:35.458146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:35.501586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239745bf-b2d9-4919-93f7-307c1aa60eb2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:35.538223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc353ca-6a39-478e-a32c-d9853ae60c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:35.538866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc353ca-6a39-478e-a32c-d9853ae60c6f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:35.539145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc353ca-6a39-478e-a32c-d9853ae60c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:35.539417 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc353ca-6a39-478e-a32c-d9853ae60c6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:35.551226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc353ca-6a39-478e-a32c-d9853ae60c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:35.552701 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc353ca-6a39-478e-a32c-d9853ae60c6f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-4fc353ca-6a39-478e-a32c-d9853ae60c6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:35.568601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:35.569609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:35.570467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:35.570467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:35.570467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:35.574443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 415/837] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:35.575851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dcc36-308b-45f5-a13f-f24f04dbce13 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:35.594960 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2138-9fe4-4907-9fb6-eca58d737206 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:35.595171 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2138-9fe4-4907-9fb6-eca58d737206 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:35.595444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f2138-9fe4-4907-9fb6-eca58d737206 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:35.597105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 416/838] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:35.604885 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591cd8b-9350-4cf1-92b3-8b716d7b05a0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:35.605368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591cd8b-9350-4cf1-92b3-8b716d7b05a0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:35.605798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591cd8b-9350-4cf1-92b3-8b716d7b05a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:35.627490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-8591cd8b-9350-4cf1-92b3-8b716d7b05a0 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:35.629294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8591cd8b-9350-4cf1-92b3-8b716d7b05a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:35.708602 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dcc36-308b-45f5-a13f-f24f04dbce13 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:35.713430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 417/839] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:35.730453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591cd8b-9350-4cf1-92b3-8b716d7b05a0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:35.748005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529364a-a3ce-448c-8a5c-dff895da5248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:35.749125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529364a-a3ce-448c-8a5c-dff895da5248 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:35.749894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529364a-a3ce-448c-8a5c-dff895da5248 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:35.750376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529364a-a3ce-448c-8a5c-dff895da5248 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:35.751238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529364a-a3ce-448c-8a5c-dff895da5248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:35.751756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529364a-a3ce-448c-8a5c-dff895da5248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:35.754919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529364a-a3ce-448c-8a5c-dff895da5248 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:35.775525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 423/840] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:35.777166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529364a-a3ce-448c-8a5c-dff895da5248 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:35.777382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:35.842812 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591cd8b-9350-4cf1-92b3-8b716d7b05a0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:35.850561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:35.882917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ad7f3-d5a5-4f37-809f-0a20b6697eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:35.883951 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ad7f3-d5a5-4f37-809f-0a20b6697eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:35.884372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ad7f3-d5a5-4f37-809f-0a20b6697eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:35.884938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ad7f3-d5a5-4f37-809f-0a20b6697eb2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:35.885750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ad7f3-d5a5-4f37-809f-0a20b6697eb2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:35.887454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ad7f3-d5a5-4f37-809f-0a20b6697eb2 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-af8ad7f3-d5a5-4f37-809f-0a20b6697eb2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:35.898334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:35.899384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 09:41:35.900275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:35.907581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529364a-a3ce-448c-8a5c-dff895da5248 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:35.908369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 418/841] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:41:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:35.909528 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ad7f3-d5a5-4f37-809f-0a20b6697eb2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:35.950029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e6c95-799e-4d9c-970a-a7a0d6efd2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:35.951071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e6c95-799e-4d9c-970a-a7a0d6efd2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:35.952332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e6c95-799e-4d9c-970a-a7a0d6efd2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:35.952776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e6c95-799e-4d9c-970a-a7a0d6efd2c3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:35.963353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e6c95-799e-4d9c-970a-a7a0d6efd2c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:35.965530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e6c95-799e-4d9c-970a-a7a0d6efd2c3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4f0e6c95-799e-4d9c-970a-a7a0d6efd2c3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:35.980326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:35.981278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:35.982195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:35.982195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:35.982195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:35.982195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:35.982195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:35.982195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:35.982195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:35.982195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:35.982195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:35.982195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:35.985086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 424/842] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Aug 11 09:41:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:35.988489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e6c95-799e-4d9c-970a-a7a0d6efd2c3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:36.042567 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ad7f3-d5a5-4f37-809f-0a20b6697eb2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:36.048129 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:36.083954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac143f30-0d95-4afa-8401-b41aaf544e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:36.086002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac143f30-0d95-4afa-8401-b41aaf544e0a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:36.086474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac143f30-0d95-4afa-8401-b41aaf544e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:36.087024 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac143f30-0d95-4afa-8401-b41aaf544e0a None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:36.087868 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac143f30-0d95-4afa-8401-b41aaf544e0a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:36.088395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac143f30-0d95-4afa-8401-b41aaf544e0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:36.090206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac143f30-0d95-4afa-8401-b41aaf544e0a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:36.100550 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 419/843] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:41:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:36.101942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac143f30-0d95-4afa-8401-b41aaf544e0a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:36.113873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e6c95-799e-4d9c-970a-a7a0d6efd2c3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:36.131536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:36.149386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d03f2-8da6-453b-8409-88fde77a75f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:36.150029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d03f2-8da6-453b-8409-88fde77a75f0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:36.150310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d03f2-8da6-453b-8409-88fde77a75f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:36.150596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d03f2-8da6-453b-8409-88fde77a75f0 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:36.151322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d03f2-8da6-453b-8409-88fde77a75f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:36.151623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d03f2-8da6-453b-8409-88fde77a75f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:36.153044 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d03f2-8da6-453b-8409-88fde77a75f0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:36.176834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 425/844] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Aug 11 09:41:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:36.178240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d03f2-8da6-453b-8409-88fde77a75f0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:36.221607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac143f30-0d95-4afa-8401-b41aaf544e0a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:36.259828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e60266-43b2-4b32-a4ea-91927eed44e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:36.260889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60266-43b2-4b32-a4ea-91927eed44e4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:36.261405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60266-43b2-4b32-a4ea-91927eed44e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:36.261901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60266-43b2-4b32-a4ea-91927eed44e4 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:36.262749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e60266-43b2-4b32-a4ea-91927eed44e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:36.263314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e60266-43b2-4b32-a4ea-91927eed44e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:36.265076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e60266-43b2-4b32-a4ea-91927eed44e4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:36.277112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 420/845] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:41:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:36.285918 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d03f2-8da6-453b-8409-88fde77a75f0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:36.326273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07ae39-a427-4241-b89b-5c2c9df52a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:36.326904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07ae39-a427-4241-b89b-5c2c9df52a15 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:36.327186 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07ae39-a427-4241-b89b-5c2c9df52a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:36.327503 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07ae39-a427-4241-b89b-5c2c9df52a15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:36.377506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:36.405210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a75b-19b6-4154-93c3-e05fc1b21472 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:36.405801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07ae39-a427-4241-b89b-5c2c9df52a15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:36.406082 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a75b-19b6-4154-93c3-e05fc1b21472 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:36.406572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a75b-19b6-4154-93c3-e05fc1b21472 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:36.407407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07ae39-a427-4241-b89b-5c2c9df52a15 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:36.408739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 421/846] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:36.414695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e124dd-048c-420e-a0fa-882df0624a5f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:36.414984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e124dd-048c-420e-a0fa-882df0624a5f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:36.415246 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e124dd-048c-420e-a0fa-882df0624a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:36.436150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-12e124dd-048c-420e-a0fa-882df0624a5f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:36.441988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-12e124dd-048c-420e-a0fa-882df0624a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:36.467595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:36.533092 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 426/848] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:41:36.537207 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 422/848] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:36.538289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07ae39-a427-4241-b89b-5c2c9df52a15 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:36.554069 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e124dd-048c-420e-a0fa-882df0624a5f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:36.644405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:36.664176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e124dd-048c-420e-a0fa-882df0624a5f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:36.665703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07ae39-a427-4241-b89b-5c2c9df52a15 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:36.700747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb1c47-1d20-48ac-8d88-6cee31b33eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:36.701499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb1c47-1d20-48ac-8d88-6cee31b33eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:36.701864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb1c47-1d20-48ac-8d88-6cee31b33eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:36.702141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb1c47-1d20-48ac-8d88-6cee31b33eb3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:36.702781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fb1c47-1d20-48ac-8d88-6cee31b33eb3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:36.703755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ab919-7186-4ea9-a7c3-cfdfa4870f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:36.704261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fb1c47-1d20-48ac-8d88-6cee31b33eb3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:36.704517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab919-7186-4ea9-a7c3-cfdfa4870f46 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:36.704909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab919-7186-4ea9-a7c3-cfdfa4870f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:36.705230 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab919-7186-4ea9-a7c3-cfdfa4870f46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-a0fb1c47-1d20-48ac-8d88-6cee31b33eb3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:36.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:36.715604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 09:41:36.716724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:36.718064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 423/849] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Aug 11 09:41:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:36.719279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ab919-7186-4ea9-a7c3-cfdfa4870f46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:36.720731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ab919-7186-4ea9-a7c3-cfdfa4870f46 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:36.725192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb1c47-1d20-48ac-8d88-6cee31b33eb3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-591ab919-7186-4ea9-a7c3-cfdfa4870f46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:36.739564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:36.740724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:36.741837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:36.741837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:36.741837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:36.743719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 427/850] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:36.748926 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ab919-7186-4ea9-a7c3-cfdfa4870f46 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:36.761927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:36.855430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb1c47-1d20-48ac-8d88-6cee31b33eb3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:36.886340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ab919-7186-4ea9-a7c3-cfdfa4870f46 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:36.902008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631b00c-777d-4637-8f9b-d02f9ae7aabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:36.902828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631b00c-777d-4637-8f9b-d02f9ae7aabd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:36.903553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631b00c-777d-4637-8f9b-d02f9ae7aabd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:36.904003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631b00c-777d-4637-8f9b-d02f9ae7aabd None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:36.905184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631b00c-777d-4637-8f9b-d02f9ae7aabd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:36.905877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631b00c-777d-4637-8f9b-d02f9ae7aabd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:36.907560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631b00c-777d-4637-8f9b-d02f9ae7aabd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:36.914251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 424/851] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Aug 11 09:41:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:36.921044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631b00c-777d-4637-8f9b-d02f9ae7aabd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:36.921186 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:36.931726 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37951bd1-91d7-43af-abd2-0c970e8e5359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:36.932894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37951bd1-91d7-43af-abd2-0c970e8e5359 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:36.933305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37951bd1-91d7-43af-abd2-0c970e8e5359 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:36.933699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37951bd1-91d7-43af-abd2-0c970e8e5359 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:36.934451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37951bd1-91d7-43af-abd2-0c970e8e5359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:36.935213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37951bd1-91d7-43af-abd2-0c970e8e5359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:36.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37951bd1-91d7-43af-abd2-0c970e8e5359 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:36.957119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 428/852] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:36.960134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37951bd1-91d7-43af-abd2-0c970e8e5359 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:37.036134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:37.046809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631b00c-777d-4637-8f9b-d02f9ae7aabd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:37.076057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37951bd1-91d7-43af-abd2-0c970e8e5359 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:37.087715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb460f-8103-49e0-a444-63fbc24597ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:37.088362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb460f-8103-49e0-a444-63fbc24597ff None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:37.088837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb460f-8103-49e0-a444-63fbc24597ff None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:37.089149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb460f-8103-49e0-a444-63fbc24597ff None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:37.089875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb460f-8103-49e0-a444-63fbc24597ff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:37.091328 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb460f-8103-49e0-a444-63fbc24597ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:37.093052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb460f-8103-49e0-a444-63fbc24597ff None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:37.105044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 425/853] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Aug 11 09:41:36 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:37.122233 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d708774-7e5c-464e-9131-af9196e79b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:37.122914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d708774-7e5c-464e-9131-af9196e79b92 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:37.123229 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d708774-7e5c-464e-9131-af9196e79b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:37.123507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d708774-7e5c-464e-9131-af9196e79b92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:37.132050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d708774-7e5c-464e-9131-af9196e79b92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:37.133625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d708774-7e5c-464e-9131-af9196e79b92 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2d708774-7e5c-464e-9131-af9196e79b92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:37.146887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:37.147823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:37.148706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:37.148706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:37.148706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:37.148706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:37.148706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:37.148706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:37.148706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:37.148706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:37.152814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 429/854] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:37.154198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb460f-8103-49e0-a444-63fbc24597ff None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:37.224068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55805ece-8047-45e7-8d66-e5c3c43408be None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:37.224400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55805ece-8047-45e7-8d66-e5c3c43408be None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:37.224675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55805ece-8047-45e7-8d66-e5c3c43408be None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:37.226546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 430/855] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:37.237138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db3515-aa86-4659-9cbf-2da3984390b3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:37.237663 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db3515-aa86-4659-9cbf-2da3984390b3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:37.238038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db3515-aa86-4659-9cbf-2da3984390b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:37.263786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-06db3515-aa86-4659-9cbf-2da3984390b3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:37.265342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-06db3515-aa86-4659-9cbf-2da3984390b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:37.284065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb460f-8103-49e0-a444-63fbc24597ff None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:37.325386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:37.325729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7332ecb-75b8-425e-8e58-17adff64fd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:37.326336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7332ecb-75b8-425e-8e58-17adff64fd81 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:37.326617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7332ecb-75b8-425e-8e58-17adff64fd81 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:37.326891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7332ecb-75b8-425e-8e58-17adff64fd81 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:37.327590 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7332ecb-75b8-425e-8e58-17adff64fd81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:37.327888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7332ecb-75b8-425e-8e58-17adff64fd81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:37.329566 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7332ecb-75b8-425e-8e58-17adff64fd81 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:37.342981 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 426/856] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:37.349306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7332ecb-75b8-425e-8e58-17adff64fd81 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:37.365036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 431/857] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:37.371109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:37.389913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db3515-aa86-4659-9cbf-2da3984390b3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:37.467991 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7332ecb-75b8-425e-8e58-17adff64fd81 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:37.506844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20a5f5-f924-4637-8f37-2859063a7ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:37.507508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20a5f5-f924-4637-8f37-2859063a7ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:37.507792 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20a5f5-f924-4637-8f37-2859063a7ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:37.508060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20a5f5-f924-4637-8f37-2859063a7ae6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:37.508785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20a5f5-f924-4637-8f37-2859063a7ae6 None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=813d8cff9d7d49d5995bfc272fee88f3) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:37.509128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20a5f5-f924-4637-8f37-2859063a7ae6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:37.510396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db3515-aa86-4659-9cbf-2da3984390b3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:37.510596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20a5f5-f924-4637-8f37-2859063a7ae6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:37.516717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 427/858] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:37 2022] GET /identity/v3/role_assignments?user.id=813d8cff9d7d49d5995bfc272fee88f3&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:37.552473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd261d18-79cc-499d-ab03-ddd7c8ae0d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:37.553497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd261d18-79cc-499d-ab03-ddd7c8ae0d82 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:37.553969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd261d18-79cc-499d-ab03-ddd7c8ae0d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:37.554395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd261d18-79cc-499d-ab03-ddd7c8ae0d82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:37.567532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd261d18-79cc-499d-ab03-ddd7c8ae0d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-cd261d18-79cc-499d-ab03-ddd7c8ae0d82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:37.572864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:37.574095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:41:37.575116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:37.577675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 432/859] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:37.583887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20a5f5-f924-4637-8f37-2859063a7ae6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:37.677733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:37.700850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20a5f5-f924-4637-8f37-2859063a7ae6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:37.735276 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22819564-4d5b-43e0-b84e-29fb3906c997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:37.735798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22819564-4d5b-43e0-b84e-29fb3906c997 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:37.736072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22819564-4d5b-43e0-b84e-29fb3906c997 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:37.736339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22819564-4d5b-43e0-b84e-29fb3906c997 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:37.737001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22819564-4d5b-43e0-b84e-29fb3906c997 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:37.737327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22819564-4d5b-43e0-b84e-29fb3906c997 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:37.739384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22819564-4d5b-43e0-b84e-29fb3906c997 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:37.747750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 428/860] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:37.752993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd261d18-79cc-499d-ab03-ddd7c8ae0d82 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:37.793179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:37.861714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd261d18-79cc-499d-ab03-ddd7c8ae0d82 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:37.898601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5489c1-0502-42c9-9404-f9c6d5d7b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:37.899297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5489c1-0502-42c9-9404-f9c6d5d7b35a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:37.899579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5489c1-0502-42c9-9404-f9c6d5d7b35a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:37.899848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5489c1-0502-42c9-9404-f9c6d5d7b35a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:37.911217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5489c1-0502-42c9-9404-f9c6d5d7b35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:37.912716 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5489c1-0502-42c9-9404-f9c6d5d7b35a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-dd5489c1-0502-42c9-9404-f9c6d5d7b35a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:37.930519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:37.931510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:37.932386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:37.932386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:37.932386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:37.934564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 433/861] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:37.939777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22819564-4d5b-43e0-b84e-29fb3906c997 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:38.041323 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22819564-4d5b-43e0-b84e-29fb3906c997 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:38.075883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9c7b1-1242-4b08-97ab-c561ec533460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:38.076496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9c7b1-1242-4b08-97ab-c561ec533460 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:38.076777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9c7b1-1242-4b08-97ab-c561ec533460 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:38.077079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9c7b1-1242-4b08-97ab-c561ec533460 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:38.077727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9c7b1-1242-4b08-97ab-c561ec533460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:38.078029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9c7b1-1242-4b08-97ab-c561ec533460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:38.079393 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9c7b1-1242-4b08-97ab-c561ec533460 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:38.089218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:38.092985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 429/862] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:38.101614 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5489c1-0502-42c9-9404-f9c6d5d7b35a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:38.208616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5489c1-0502-42c9-9404-f9c6d5d7b35a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:38.229305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:38.244220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce7a1d-e4c8-4766-9a45-b257c4a3c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:38.244853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7a1d-e4c8-4766-9a45-b257c4a3c95f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:38.245165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7a1d-e4c8-4766-9a45-b257c4a3c95f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:38.245442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7a1d-e4c8-4766-9a45-b257c4a3c95f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:38.253275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce7a1d-e4c8-4766-9a45-b257c4a3c95f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:38.254693 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce7a1d-e4c8-4766-9a45-b257c4a3c95f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-56ce7a1d-e4c8-4766-9a45-b257c4a3c95f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:38.274346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:38.275342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:38.276270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:38.276270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:38.276270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:38.276270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:38.276270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:38.276270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:38.276270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:38.276270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:38.276270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 09:41:38.276270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:38.279009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 434/863] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:41:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:38.285783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9c7b1-1242-4b08-97ab-c561ec533460 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:38.389180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9c7b1-1242-4b08-97ab-c561ec533460 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:38.423996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77cb7ac-1d86-4a21-b021-12fc3f5f917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:38.424606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cb7ac-1d86-4a21-b021-12fc3f5f917c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:38.424886 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cb7ac-1d86-4a21-b021-12fc3f5f917c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:38.425314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cb7ac-1d86-4a21-b021-12fc3f5f917c None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:38.426210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cb7ac-1d86-4a21-b021-12fc3f5f917c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:38.426514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cb7ac-1d86-4a21-b021-12fc3f5f917c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:38.427931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77cb7ac-1d86-4a21-b021-12fc3f5f917c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:38.446664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 430/864] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Aug 11 09:41:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:38.452267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce7a1d-e4c8-4766-9a45-b257c4a3c95f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:38.514298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8abed-da44-424c-8723-772e84c15263 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:38.514940 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8abed-da44-424c-8723-772e84c15263 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:38.515536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8abed-da44-424c-8723-772e84c15263 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:38.517422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 431/865] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:38.523469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac48625-ed15-493b-8669-37f8884af8c7 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:38.523979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac48625-ed15-493b-8669-37f8884af8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:38.524487 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac48625-ed15-493b-8669-37f8884af8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:38.543524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-cac48625-ed15-493b-8669-37f8884af8c7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:38.545116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cac48625-ed15-493b-8669-37f8884af8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:38.566749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce7a1d-e4c8-4766-9a45-b257c4a3c95f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:38.603404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:38.610895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19add9-300c-45ec-9c31-3eb07ae76eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:38.612318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19add9-300c-45ec-9c31-3eb07ae76eed None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:38.613470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19add9-300c-45ec-9c31-3eb07ae76eed None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:38.614025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19add9-300c-45ec-9c31-3eb07ae76eed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:38.626242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 432/866] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:38.644950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac48625-ed15-493b-8669-37f8884af8c7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:38.645897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:38.713366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19add9-300c-45ec-9c31-3eb07ae76eed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:38.715616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19add9-300c-45ec-9c31-3eb07ae76eed None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:38.772997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac48625-ed15-493b-8669-37f8884af8c7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:38.814513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6823f7-59f8-4af9-9de2-88a221957289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:38.815588 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6823f7-59f8-4af9-9de2-88a221957289 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:38.816153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6823f7-59f8-4af9-9de2-88a221957289 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:38.816626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6823f7-59f8-4af9-9de2-88a221957289 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:38.817774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6823f7-59f8-4af9-9de2-88a221957289 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:38.819512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6823f7-59f8-4af9-9de2-88a221957289 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:38.843642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 435/867] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:41:38.845086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19add9-300c-45ec-9c31-3eb07ae76eed None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:38.898056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:38.921579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 433/868] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:41:38 2022] POST /identity/v3/services => generated 251 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:38.967902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19add9-300c-45ec-9c31-3eb07ae76eed None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:39.013119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca38c0-d441-468a-a2c3-271b09455d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:39.013825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca38c0-d441-468a-a2c3-271b09455d8a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:39.014110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca38c0-d441-468a-a2c3-271b09455d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:39.014383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca38c0-d441-468a-a2c3-271b09455d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:39.026583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca38c0-d441-468a-a2c3-271b09455d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:39.028077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca38c0-d441-468a-a2c3-271b09455d8a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e8ca38c0-d441-468a-a2c3-271b09455d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:39.044133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:39.045148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:39.046030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:39.046030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:39.046030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:39.046030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 436/869] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:39.053056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6823f7-59f8-4af9-9de2-88a221957289 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:39.108970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:39.164374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6823f7-59f8-4af9-9de2-88a221957289 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:39.205430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f656b-1176-4582-bf6f-9689b92358bd None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:39.205623 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a795837-4d3f-4031-816f-16e5397a4a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:39.205718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f656b-1176-4582-bf6f-9689b92358bd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:39.206026 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f656b-1176-4582-bf6f-9689b92358bd None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:39.206908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a795837-4d3f-4031-816f-16e5397a4a58 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:39.207409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a795837-4d3f-4031-816f-16e5397a4a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:39.207805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a795837-4d3f-4031-816f-16e5397a4a58 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:39.208210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 437/870] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:39.208472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a795837-4d3f-4031-816f-16e5397a4a58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:39.208783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a795837-4d3f-4031-816f-16e5397a4a58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:39.210250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a795837-4d3f-4031-816f-16e5397a4a58 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:39.214248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324aa43-49cc-4e1b-8335-47c2fd1534aa None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:39.215138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324aa43-49cc-4e1b-8335-47c2fd1534aa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:39.215621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324aa43-49cc-4e1b-8335-47c2fd1534aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:39.224600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 434/871] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:39.231164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a795837-4d3f-4031-816f-16e5397a4a58 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:39.240008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5324aa43-49cc-4e1b-8335-47c2fd1534aa None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:39.241626 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5324aa43-49cc-4e1b-8335-47c2fd1534aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:39.325035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 438/872] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:39 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:39.339603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324aa43-49cc-4e1b-8335-47c2fd1534aa None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:39.358849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a795837-4d3f-4031-816f-16e5397a4a58 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:39.368393 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:39.372092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:39.404919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e024edc-0908-42a1-98bd-5a8f973723a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:39.405769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e024edc-0908-42a1-98bd-5a8f973723a5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:39.406049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e024edc-0908-42a1-98bd-5a8f973723a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:39.406319 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e024edc-0908-42a1-98bd-5a8f973723a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:39.422225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e024edc-0908-42a1-98bd-5a8f973723a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:39.422225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e024edc-0908-42a1-98bd-5a8f973723a5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-4e024edc-0908-42a1-98bd-5a8f973723a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:39.436146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:39.437064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:39.437957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:39.437957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:39.437957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:39.437957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:39.437957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:39.437957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:39.437957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:39.437957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:39.446359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 435/873] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:39.448177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e024edc-0908-42a1-98bd-5a8f973723a5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:39.455768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324aa43-49cc-4e1b-8335-47c2fd1534aa None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:39.492982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b63014-5ed5-4758-aa80-7b94a1134c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:39.493656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b63014-5ed5-4758-aa80-7b94a1134c43 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:39.493947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b63014-5ed5-4758-aa80-7b94a1134c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:39.494252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b63014-5ed5-4758-aa80-7b94a1134c43 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:39.494907 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b63014-5ed5-4758-aa80-7b94a1134c43 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:39.496257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b63014-5ed5-4758-aa80-7b94a1134c43 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:39.569313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e024edc-0908-42a1-98bd-5a8f973723a5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:39.610951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 439/874] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:41:39 2022] POST /identity/v3/services => generated 269 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:41:39.617306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b4cb0-a186-42d2-8c9d-b18927684e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:39.618309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4cb0-a186-42d2-8c9d-b18927684e2b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:39.618760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4cb0-a186-42d2-8c9d-b18927684e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:39.619208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4cb0-a186-42d2-8c9d-b18927684e2b None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:39.620131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b4cb0-a186-42d2-8c9d-b18927684e2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:39.620614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b4cb0-a186-42d2-8c9d-b18927684e2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:39.622371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b4cb0-a186-42d2-8c9d-b18927684e2b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:39.637565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 436/875] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:39.645041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b63014-5ed5-4758-aa80-7b94a1134c43 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:39.731340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734814a2-d27e-41af-b81d-83274c10e750 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:39.731987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734814a2-d27e-41af-b81d-83274c10e750 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:39.732472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734814a2-d27e-41af-b81d-83274c10e750 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:39.734425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 437/876] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:39.741233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371806-9f04-4cd7-be53-1cc10321347f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:39.741873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371806-9f04-4cd7-be53-1cc10321347f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:39.742361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371806-9f04-4cd7-be53-1cc10321347f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:39.754694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:39.762917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-29371806-9f04-4cd7-be53-1cc10321347f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:39.764156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-29371806-9f04-4cd7-be53-1cc10321347f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:39.777806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b63014-5ed5-4758-aa80-7b94a1134c43 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:39.787168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:39.824886 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3834763-a350-4c13-912b-cd2ac8f09cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:39.825949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3834763-a350-4c13-912b-cd2ac8f09cef None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:39.826439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3834763-a350-4c13-912b-cd2ac8f09cef None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:39.827016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3834763-a350-4c13-912b-cd2ac8f09cef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:39.828942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3834763-a350-4c13-912b-cd2ac8f09cef None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=915ee44218064af896020d008ff31957) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:39.829501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3834763-a350-4c13-912b-cd2ac8f09cef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:39.831353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3834763-a350-4c13-912b-cd2ac8f09cef None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:39.838494 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 440/877] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:39 2022] GET /identity/v3/role_assignments?user.id=915ee44218064af896020d008ff31957&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:39.857459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 438/878] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:39.881891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3834763-a350-4c13-912b-cd2ac8f09cef None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:40.008668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3834763-a350-4c13-912b-cd2ac8f09cef None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:40.045431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe86b7-030f-41b6-8753-253b7ffc0aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:40.046068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe86b7-030f-41b6-8753-253b7ffc0aaf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:40.046352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe86b7-030f-41b6-8753-253b7ffc0aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:40.046627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe86b7-030f-41b6-8753-253b7ffc0aaf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:40.057515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe86b7-030f-41b6-8753-253b7ffc0aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-adfe86b7-030f-41b6-8753-253b7ffc0aaf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:40.062955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:40.063875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:40.064692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 441/879] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Aug 11 09:41:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:40.072568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29371806-9f04-4cd7-be53-1cc10321347f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:40.186300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29371806-9f04-4cd7-be53-1cc10321347f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:40.229842 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72c7f1-9f1d-4a8c-8cae-164d8eae9037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:40.230446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72c7f1-9f1d-4a8c-8cae-164d8eae9037 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:40.230722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72c7f1-9f1d-4a8c-8cae-164d8eae9037 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:40.230988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72c7f1-9f1d-4a8c-8cae-164d8eae9037 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:40.231647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72c7f1-9f1d-4a8c-8cae-164d8eae9037 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:40.231944 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72c7f1-9f1d-4a8c-8cae-164d8eae9037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:40.233325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72c7f1-9f1d-4a8c-8cae-164d8eae9037 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:40.242101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 439/880] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Aug 11 09:41:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:40.247331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe86b7-030f-41b6-8753-253b7ffc0aaf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:40.261329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:40.355963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe86b7-030f-41b6-8753-253b7ffc0aaf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:40.392791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf0e8c-3793-4aab-9f14-93ccf1e65f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:40.394765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0e8c-3793-4aab-9f14-93ccf1e65f47 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:40.395389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0e8c-3793-4aab-9f14-93ccf1e65f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:40.395901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf0e8c-3793-4aab-9f14-93ccf1e65f47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:40.408461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf0e8c-3793-4aab-9f14-93ccf1e65f47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:40.411351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf0e8c-3793-4aab-9f14-93ccf1e65f47 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-cbcf0e8c-3793-4aab-9f14-93ccf1e65f47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:40.426891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:40.427812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:40.428730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:40.428730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:40.428730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:40.430832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 442/881] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:40.438104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72c7f1-9f1d-4a8c-8cae-164d8eae9037 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:40.460713 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:40.544581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72c7f1-9f1d-4a8c-8cae-164d8eae9037 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:40.586659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b764c-af8c-4b0e-9746-9b99419a32a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:40.587261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b764c-af8c-4b0e-9746-9b99419a32a7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:40.587542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b764c-af8c-4b0e-9746-9b99419a32a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:40.587809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b764c-af8c-4b0e-9746-9b99419a32a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:40.588451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b764c-af8c-4b0e-9746-9b99419a32a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:40.588747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b764c-af8c-4b0e-9746-9b99419a32a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:40.590186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b764c-af8c-4b0e-9746-9b99419a32a7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:40.602645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 440/882] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:40.607910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf0e8c-3793-4aab-9f14-93ccf1e65f47 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:40.672767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:40.714458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf0e8c-3793-4aab-9f14-93ccf1e65f47 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:40.750266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb786-511f-44a4-bf72-c152841ced2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:40.750903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb786-511f-44a4-bf72-c152841ced2c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:40.751182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb786-511f-44a4-bf72-c152841ced2c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:40.751454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb786-511f-44a4-bf72-c152841ced2c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:40.759431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bb786-511f-44a4-bf72-c152841ced2c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:40.760855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bb786-511f-44a4-bf72-c152841ced2c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-948bb786-511f-44a4-bf72-c152841ced2c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:40.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:40.773395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:40.774319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:40.774319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:40.774319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:40.774319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:40.774319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:40.774319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:40.774319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:40.774319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:40.774319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:40.774319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:40.774319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 443/883] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Aug 11 09:41:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:40.782085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b764c-af8c-4b0e-9746-9b99419a32a7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:40.865270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:40.887608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b764c-af8c-4b0e-9746-9b99419a32a7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:40.927864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc33618-401c-45aa-a68c-874485c7e95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:40.928472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc33618-401c-45aa-a68c-874485c7e95c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:40.928750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc33618-401c-45aa-a68c-874485c7e95c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:40.929037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc33618-401c-45aa-a68c-874485c7e95c None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:40.929780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc33618-401c-45aa-a68c-874485c7e95c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:40.930083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc33618-401c-45aa-a68c-874485c7e95c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:40.931499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc33618-401c-45aa-a68c-874485c7e95c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:40.949468 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 441/884] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Aug 11 09:41:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:40.954828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb786-511f-44a4-bf72-c152841ced2c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:41.060177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb786-511f-44a4-bf72-c152841ced2c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:41.086499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:41.098317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd040f54-f0de-4a69-a5b1-8cbe94f10fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:41.098929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd040f54-f0de-4a69-a5b1-8cbe94f10fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.099208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd040f54-f0de-4a69-a5b1-8cbe94f10fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.099479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd040f54-f0de-4a69-a5b1-8cbe94f10fe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.133163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68de9-8b3c-4b03-9cf1-9be1a4dcd00e None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.133659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68de9-8b3c-4b03-9cf1-9be1a4dcd00e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.133659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f68de9-8b3c-4b03-9cf1-9be1a4dcd00e None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.134924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 442/885] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:41.142145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe9207-1c03-4688-8540-6bf3d7aa96cf None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.142426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe9207-1c03-4688-8540-6bf3d7aa96cf None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.142698 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe9207-1c03-4688-8540-6bf3d7aa96cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.163297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-dcfe9207-1c03-4688-8540-6bf3d7aa96cf None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:41.164504 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dcfe9207-1c03-4688-8540-6bf3d7aa96cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:41.172483 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd040f54-f0de-4a69-a5b1-8cbe94f10fe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:41.173985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd040f54-f0de-4a69-a5b1-8cbe94f10fe2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:41.241673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 443/886] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:41.264091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe9207-1c03-4688-8540-6bf3d7aa96cf None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:41.309034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 444/887] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:40 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:41:41.314664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd040f54-f0de-4a69-a5b1-8cbe94f10fe2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:41.339418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:41.341874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:41.382602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe9207-1c03-4688-8540-6bf3d7aa96cf None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:41.417638 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd040f54-f0de-4a69-a5b1-8cbe94f10fe2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:41.420825 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ee7a1-d6ae-4150-9e40-62c955e9eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:41.421781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ee7a1-d6ae-4150-9e40-62c955e9eea4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.422262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ee7a1-d6ae-4150-9e40-62c955e9eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.422757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ee7a1-d6ae-4150-9e40-62c955e9eea4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.423549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ee7a1-d6ae-4150-9e40-62c955e9eea4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:41.425316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ee7a1-d6ae-4150-9e40-62c955e9eea4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-444ee7a1-d6ae-4150-9e40-62c955e9eea4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:41.435560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:41.436546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 09:41:41.437377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:41.446335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 444/888] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:41:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:41.447382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ee7a1-d6ae-4150-9e40-62c955e9eea4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:41.464027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d65dc9-6663-48ed-a6ad-9e9fe3656223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:41.464673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d65dc9-6663-48ed-a6ad-9e9fe3656223 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.465041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d65dc9-6663-48ed-a6ad-9e9fe3656223 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.465318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d65dc9-6663-48ed-a6ad-9e9fe3656223 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.478827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d65dc9-6663-48ed-a6ad-9e9fe3656223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:41.480531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d65dc9-6663-48ed-a6ad-9e9fe3656223 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-26d65dc9-6663-48ed-a6ad-9e9fe3656223 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:41.499724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:41.500709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:41.501640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:41.501640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:41.501640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:41.505087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 445/889] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:41.507053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d65dc9-6663-48ed-a6ad-9e9fe3656223 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:41.584593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ee7a1-d6ae-4150-9e40-62c955e9eea4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:41.609849 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:41.614739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:41.623297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4d599-3839-4372-9e6f-f6d48e101d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:41.624260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4d599-3839-4372-9e6f-f6d48e101d89 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.624699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4d599-3839-4372-9e6f-f6d48e101d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.625192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4d599-3839-4372-9e6f-f6d48e101d89 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.625983 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4d599-3839-4372-9e6f-f6d48e101d89 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:41.626438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4d599-3839-4372-9e6f-f6d48e101d89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:41.628089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4d599-3839-4372-9e6f-f6d48e101d89 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:41.633843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 445/890] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:41:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:41.640127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4d599-3839-4372-9e6f-f6d48e101d89 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:41.644498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d65dc9-6663-48ed-a6ad-9e9fe3656223 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:41.682376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffdb3a5-8a15-4417-8e8c-80f48eddf70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:41.682376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdb3a5-8a15-4417-8e8c-80f48eddf70f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.682629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdb3a5-8a15-4417-8e8c-80f48eddf70f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.682860 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdb3a5-8a15-4417-8e8c-80f48eddf70f None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.683478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffdb3a5-8a15-4417-8e8c-80f48eddf70f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:41.683780 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffdb3a5-8a15-4417-8e8c-80f48eddf70f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:41.685143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffdb3a5-8a15-4417-8e8c-80f48eddf70f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:41.699146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 446/891] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:41.704296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffdb3a5-8a15-4417-8e8c-80f48eddf70f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:41.756037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4d599-3839-4372-9e6f-f6d48e101d89 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:41.793183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dd407-0216-43ee-9dfd-190bd3b191db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:41.794185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dd407-0216-43ee-9dfd-190bd3b191db None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.794677 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dd407-0216-43ee-9dfd-190bd3b191db None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.795161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dd407-0216-43ee-9dfd-190bd3b191db None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.796044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85dd407-0216-43ee-9dfd-190bd3b191db None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:41.796613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85dd407-0216-43ee-9dfd-190bd3b191db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:41.798195 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85dd407-0216-43ee-9dfd-190bd3b191db None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:41.808775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 446/892] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:41:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:41.810287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dd407-0216-43ee-9dfd-190bd3b191db None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:41.815537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffdb3a5-8a15-4417-8e8c-80f48eddf70f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:41.854461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a755c-f097-4e95-b746-5441d0c3685e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:41.855098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a755c-f097-4e95-b746-5441d0c3685e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.855378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a755c-f097-4e95-b746-5441d0c3685e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.855711 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a755c-f097-4e95-b746-5441d0c3685e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.857837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:41.863826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a755c-f097-4e95-b746-5441d0c3685e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:41.865407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a755c-f097-4e95-b746-5441d0c3685e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:41.876257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e74a755c-f097-4e95-b746-5441d0c3685e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:41.879513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:41.880464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:41.881380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:41.881380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:41.881380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:41.881380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:41.881380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:41.881380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:41.881380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:41.881380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:41.881380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 447/893] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:41.881862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2d1df-b095-459a-b7d3-3addd8ecb75a None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.882780 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2d1df-b095-459a-b7d3-3addd8ecb75a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.882780 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2d1df-b095-459a-b7d3-3addd8ecb75a None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.883720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 448/894] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:41.888127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2d1df-b095-459a-b7d3-3addd8ecb75a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:41.929213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dd407-0216-43ee-9dfd-190bd3b191db None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:41.964455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b526ce-b5ea-48bc-87ed-1246d2b9c446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:41.965005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b526ce-b5ea-48bc-87ed-1246d2b9c446 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.965298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b526ce-b5ea-48bc-87ed-1246d2b9c446 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.965570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b526ce-b5ea-48bc-87ed-1246d2b9c446 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.966271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b526ce-b5ea-48bc-87ed-1246d2b9c446 None admin] RBAC: Adding query filter params (interface=public, service_id=22f2c31251a44d55b9b29467375d08a9, region_id=RegionOne) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:41.966578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b526ce-b5ea-48bc-87ed-1246d2b9c446 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:41.967932 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b526ce-b5ea-48bc-87ed-1246d2b9c446 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:41.975393 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 447/895] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Aug 11 09:41:41 2022] GET /identity/v3/endpoints?service_id=22f2c31251a44d55b9b29467375d08a9&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:41.976581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08f4d0-26de-4d0d-8db1-ae1d84fbdec4 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:41.976860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08f4d0-26de-4d0d-8db1-ae1d84fbdec4 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:41.977237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08f4d0-26de-4d0d-8db1-ae1d84fbdec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:41.995683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1f08f4d0-26de-4d0d-8db1-ae1d84fbdec4 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:41.996892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1f08f4d0-26de-4d0d-8db1-ae1d84fbdec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:42.007311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2d1df-b095-459a-b7d3-3addd8ecb75a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:42.041292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94a59e-95a4-413b-8c22-404566f65769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:42.041894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94a59e-95a4-413b-8c22-404566f65769 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.042171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94a59e-95a4-413b-8c22-404566f65769 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.042439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94a59e-95a4-413b-8c22-404566f65769 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.043124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94a59e-95a4-413b-8c22-404566f65769 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:42.043429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94a59e-95a4-413b-8c22-404566f65769 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:42.044831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94a59e-95a4-413b-8c22-404566f65769 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:42.056150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 449/896] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:42.056150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94a59e-95a4-413b-8c22-404566f65769 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:42.062729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 448/897] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:42.063971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603acb25-d16f-40ca-b962-c4f0a41ea39e None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.064257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603acb25-d16f-40ca-b962-c4f0a41ea39e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.064528 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603acb25-d16f-40ca-b962-c4f0a41ea39e None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.066613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 449/898] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:42.067402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603acb25-d16f-40ca-b962-c4f0a41ea39e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:42.118593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:42.123509 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:42.162835 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94a59e-95a4-413b-8c22-404566f65769 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:42.172578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603acb25-d16f-40ca-b962-c4f0a41ea39e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:42.197264 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6f237-a0ec-4ddd-88ee-25a95c3e8e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:42.197876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6f237-a0ec-4ddd-88ee-25a95c3e8e3f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.198153 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6f237-a0ec-4ddd-88ee-25a95c3e8e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.198425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6f237-a0ec-4ddd-88ee-25a95c3e8e3f None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.199024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6f237-a0ec-4ddd-88ee-25a95c3e8e3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:42.200365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6f237-a0ec-4ddd-88ee-25a95c3e8e3f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:42.206005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 450/899] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Aug 11 09:41:42 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:41:42.207719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d90f5-5bae-4484-b829-b6cd13ecba5a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.208001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d90f5-5bae-4484-b829-b6cd13ecba5a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.208271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d90f5-5bae-4484-b829-b6cd13ecba5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.223238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd91646-b6c0-41c3-a6a5-9f9a0bdd49ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:42.223937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd91646-b6c0-41c3-a6a5-9f9a0bdd49ab None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.224308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd91646-b6c0-41c3-a6a5-9f9a0bdd49ab None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.224651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd91646-b6c0-41c3-a6a5-9f9a0bdd49ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.225508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd91646-b6c0-41c3-a6a5-9f9a0bdd49ab None admin] RBAC: Adding query filter params (role.id=1a335dba62e246ec820c70f60d2558b9, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=813d8cff9d7d49d5995bfc272fee88f3) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:42.225881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd91646-b6c0-41c3-a6a5-9f9a0bdd49ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:42.227467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd91646-b6c0-41c3-a6a5-9f9a0bdd49ab None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:42.227721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-944d90f5-5bae-4484-b829-b6cd13ecba5a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:42.228960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-944d90f5-5bae-4484-b829-b6cd13ecba5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:42.235305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 450/900] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:42 2022] GET /identity/v3/role_assignments?user.id=813d8cff9d7d49d5995bfc272fee88f3&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=1a335dba62e246ec820c70f60d2558b9 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:42.236084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd91646-b6c0-41c3-a6a5-9f9a0bdd49ab None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:42.316857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 451/901] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:42.338999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d90f5-5bae-4484-b829-b6cd13ecba5a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:42.370489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd91646-b6c0-41c3-a6a5-9f9a0bdd49ab None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:42.387077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:42.411014 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:42.412204 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b681c-2d8d-41f0-b037-95d36377b8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:42.412871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b681c-2d8d-41f0-b037-95d36377b8ed None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.413211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b681c-2d8d-41f0-b037-95d36377b8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.413476 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b681c-2d8d-41f0-b037-95d36377b8ed None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.414084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b681c-2d8d-41f0-b037-95d36377b8ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:42.415577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b681c-2d8d-41f0-b037-95d36377b8ed None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-389b681c-2d8d-41f0-b037-95d36377b8ed None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:42.426154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:42.427133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 09:41:42.428077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:42.428835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 451/902] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Aug 11 09:41:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:42.434240 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b681c-2d8d-41f0-b037-95d36377b8ed None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:42.480468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d90f5-5bae-4484-b829-b6cd13ecba5a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:42.533048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5fcf0-70f5-4f41-8555-7ffd88006141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:42.533696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5fcf0-70f5-4f41-8555-7ffd88006141 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.534159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5fcf0-70f5-4f41-8555-7ffd88006141 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.535136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab5fcf0-70f5-4f41-8555-7ffd88006141 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.535807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5fcf0-70f5-4f41-8555-7ffd88006141 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:42.537562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab5fcf0-70f5-4f41-8555-7ffd88006141 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6ab5fcf0-70f5-4f41-8555-7ffd88006141 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:42.550960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:42.551982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:42.552788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 452/903] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Aug 11 09:41:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:42.558262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5fcf0-70f5-4f41-8555-7ffd88006141 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:42.568149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b681c-2d8d-41f0-b037-95d36377b8ed None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:42.604032 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c36a31-2528-41ab-8210-7aea72c492aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:42.604919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c36a31-2528-41ab-8210-7aea72c492aa None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.605370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c36a31-2528-41ab-8210-7aea72c492aa None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.605790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c36a31-2528-41ab-8210-7aea72c492aa None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.606628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c36a31-2528-41ab-8210-7aea72c492aa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:42.607096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c36a31-2528-41ab-8210-7aea72c492aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:42.608610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c36a31-2528-41ab-8210-7aea72c492aa None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:42.614625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 452/904] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Aug 11 09:41:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:42.622714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c36a31-2528-41ab-8210-7aea72c492aa None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:42.666797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab5fcf0-70f5-4f41-8555-7ffd88006141 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:42.670071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:42.680787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:42.703487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d32d8-c264-46aa-9496-d2d9016c715b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:42.704108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d32d8-c264-46aa-9496-d2d9016c715b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.704392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d32d8-c264-46aa-9496-d2d9016c715b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.704671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d32d8-c264-46aa-9496-d2d9016c715b None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.705360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d32d8-c264-46aa-9496-d2d9016c715b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:42.705670 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d32d8-c264-46aa-9496-d2d9016c715b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:42.707063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d32d8-c264-46aa-9496-d2d9016c715b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:42.712747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 453/905] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Aug 11 09:41:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:42.718450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d32d8-c264-46aa-9496-d2d9016c715b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:42.739474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c36a31-2528-41ab-8210-7aea72c492aa None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:42.782700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14dc3b-86c4-4280-85c8-e64656f02891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:42.784049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14dc3b-86c4-4280-85c8-e64656f02891 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.785299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14dc3b-86c4-4280-85c8-e64656f02891 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.785843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14dc3b-86c4-4280-85c8-e64656f02891 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.786630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14dc3b-86c4-4280-85c8-e64656f02891 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:42.787235 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14dc3b-86c4-4280-85c8-e64656f02891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:42.789444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14dc3b-86c4-4280-85c8-e64656f02891 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:42.796437 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 453/906] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Aug 11 09:41:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:42.805056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14dc3b-86c4-4280-85c8-e64656f02891 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:42.849684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d32d8-c264-46aa-9496-d2d9016c715b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:42.896124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514fd74-864d-43a4-ac6f-fcc0a578646b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:42.896852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514fd74-864d-43a4-ac6f-fcc0a578646b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.897224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514fd74-864d-43a4-ac6f-fcc0a578646b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.897573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514fd74-864d-43a4-ac6f-fcc0a578646b None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.898648 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514fd74-864d-43a4-ac6f-fcc0a578646b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:42.898960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514fd74-864d-43a4-ac6f-fcc0a578646b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:42.900748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514fd74-864d-43a4-ac6f-fcc0a578646b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:42.907490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 454/907] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Aug 11 09:41:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:42.915641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14dc3b-86c4-4280-85c8-e64656f02891 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:42.924482 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:42.957900 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540dd318-647d-4a8b-bc5b-d5caa30649d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:42.958620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540dd318-647d-4a8b-bc5b-d5caa30649d3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:42.958969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540dd318-647d-4a8b-bc5b-d5caa30649d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:42.959310 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540dd318-647d-4a8b-bc5b-d5caa30649d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:42.960124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540dd318-647d-4a8b-bc5b-d5caa30649d3 None admin] RBAC: Adding query filter params (interface=public, service_id=351ece2c92f249ccb121c33cbd36b52c, region_id=RegionOne) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:42.960501 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540dd318-647d-4a8b-bc5b-d5caa30649d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:42.962032 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540dd318-647d-4a8b-bc5b-d5caa30649d3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:42.968674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 454/908] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Aug 11 09:41:42 2022] GET /identity/v3/endpoints?service_id=351ece2c92f249ccb121c33cbd36b52c&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:42.975191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514fd74-864d-43a4-ac6f-fcc0a578646b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:43.026302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:43.093320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514fd74-864d-43a4-ac6f-fcc0a578646b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:43.130266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f8886-5323-4b75-8124-d5c99051c785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:43.130881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f8886-5323-4b75-8124-d5c99051c785 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:43.131164 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f8886-5323-4b75-8124-d5c99051c785 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:43.131444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f8886-5323-4b75-8124-d5c99051c785 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:43.132111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f8886-5323-4b75-8124-d5c99051c785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:43.133561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f8886-5323-4b75-8124-d5c99051c785 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:43.139737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 455/909] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Aug 11 09:41:42 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:41:44.361326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e0ff2-f0fd-402a-bc82-e0129f68f368 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:44.361867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e0ff2-f0fd-402a-bc82-e0129f68f368 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:44.361867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e0ff2-f0fd-402a-bc82-e0129f68f368 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:44.363450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 455/910] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:44.368837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2ab03-bcd2-4abf-b5e6-0c67e51045a9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:44.369693 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2ab03-bcd2-4abf-b5e6-0c67e51045a9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:44.370195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2ab03-bcd2-4abf-b5e6-0c67e51045a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:44.372584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81511d1-cfc7-4642-80cf-931da5c0cfcc None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:44.372866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81511d1-cfc7-4642-80cf-931da5c0cfcc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:44.373166 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81511d1-cfc7-4642-80cf-931da5c0cfcc None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:44.374872 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 456/911] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:44.381756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dd46d-b536-45ff-83a1-305af1f72ee2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:44.387295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dd46d-b536-45ff-83a1-305af1f72ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:44.387295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dd46d-b536-45ff-83a1-305af1f72ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:44.393477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f9c2ab03-bcd2-4abf-b5e6-0c67e51045a9 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:44.395117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f9c2ab03-bcd2-4abf-b5e6-0c67e51045a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:44.407301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-566dd46d-b536-45ff-83a1-305af1f72ee2 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:44.408592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-566dd46d-b536-45ff-83a1-305af1f72ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:44.478448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 456/912] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:44.497301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 457/913] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:44.501307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2ab03-bcd2-4abf-b5e6-0c67e51045a9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:44.520323 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dd46d-b536-45ff-83a1-305af1f72ee2 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:44.541076 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:44.622025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2ab03-bcd2-4abf-b5e6-0c67e51045a9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:44.641422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:44.645361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dd46d-b536-45ff-83a1-305af1f72ee2 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:44.662008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242416b-9863-4372-be8e-06a1c6eb8985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:44.662649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242416b-9863-4372-be8e-06a1c6eb8985 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:44.662923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242416b-9863-4372-be8e-06a1c6eb8985 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:44.663194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242416b-9863-4372-be8e-06a1c6eb8985 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:44.663825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242416b-9863-4372-be8e-06a1c6eb8985 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:44.665273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242416b-9863-4372-be8e-06a1c6eb8985 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6242416b-9863-4372-be8e-06a1c6eb8985 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:44.675825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:44.676831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 09:41:44.677687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:44.681145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 457/914] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:41:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:44.683482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242416b-9863-4372-be8e-06a1c6eb8985 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:44.688139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7bc7c-bcc9-4e81-a7f0-f31977abdae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:44.689033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7bc7c-bcc9-4e81-a7f0-f31977abdae8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:44.689511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7bc7c-bcc9-4e81-a7f0-f31977abdae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:44.690069 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7bc7c-bcc9-4e81-a7f0-f31977abdae8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:44.701895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7bc7c-bcc9-4e81-a7f0-f31977abdae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-3bf7bc7c-bcc9-4e81-a7f0-f31977abdae8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:44.706986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:44.707878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:41:44.708689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:44.715919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 458/915] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Aug 11 09:41:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:44.716964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7bc7c-bcc9-4e81-a7f0-f31977abdae8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:44.791395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:44.795727 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242416b-9863-4372-be8e-06a1c6eb8985 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:44.823961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7bc7c-bcc9-4e81-a7f0-f31977abdae8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:44.834643 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f17811-82bd-4196-ba61-15ef6e7da0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:44.835268 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f17811-82bd-4196-ba61-15ef6e7da0cf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:44.835549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f17811-82bd-4196-ba61-15ef6e7da0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:44.835824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f17811-82bd-4196-ba61-15ef6e7da0cf None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:44.836690 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f17811-82bd-4196-ba61-15ef6e7da0cf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:44.836887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f17811-82bd-4196-ba61-15ef6e7da0cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:44.838277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f17811-82bd-4196-ba61-15ef6e7da0cf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:44.848510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 458/916] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:41:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:44.849896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f17811-82bd-4196-ba61-15ef6e7da0cf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:44.862799 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa55f8-f405-49a1-8214-8d1aa509a920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:44.863896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa55f8-f405-49a1-8214-8d1aa509a920 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:44.864382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa55f8-f405-49a1-8214-8d1aa509a920 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:44.864848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa55f8-f405-49a1-8214-8d1aa509a920 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:44.865744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa55f8-f405-49a1-8214-8d1aa509a920 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:44.866256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa55f8-f405-49a1-8214-8d1aa509a920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:44.867922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa55f8-f405-49a1-8214-8d1aa509a920 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:44.882034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 459/917] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Aug 11 09:41:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:44.883400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa55f8-f405-49a1-8214-8d1aa509a920 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:44.910244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:44.963761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f17811-82bd-4196-ba61-15ef6e7da0cf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:44.994884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa55f8-f405-49a1-8214-8d1aa509a920 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:45.002529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bad20-26ee-4eb4-89c4-ccb05207c7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:45.003698 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bad20-26ee-4eb4-89c4-ccb05207c7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.003982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bad20-26ee-4eb4-89c4-ccb05207c7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.004374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bad20-26ee-4eb4-89c4-ccb05207c7c3 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.005029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bad20-26ee-4eb4-89c4-ccb05207c7c3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:45.005343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bad20-26ee-4eb4-89c4-ccb05207c7c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:45.006825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bad20-26ee-4eb4-89c4-ccb05207c7c3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:45.012360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 459/918] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Aug 11 09:41:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:45.014329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637dad8-1d2f-42b2-a07b-14c8409248d7 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.014847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637dad8-1d2f-42b2-a07b-14c8409248d7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.015334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637dad8-1d2f-42b2-a07b-14c8409248d7 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.016692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 460/919] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:45.018011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637dad8-1d2f-42b2-a07b-14c8409248d7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:45.035859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848e03d-c0a1-448f-aef1-3815d5500893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:45.036484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848e03d-c0a1-448f-aef1-3815d5500893 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.036764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848e03d-c0a1-448f-aef1-3815d5500893 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.037066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848e03d-c0a1-448f-aef1-3815d5500893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.048813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848e03d-c0a1-448f-aef1-3815d5500893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:45.050803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848e03d-c0a1-448f-aef1-3815d5500893 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:45.053886 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9848e03d-c0a1-448f-aef1-3815d5500893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:45.068663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:45.069928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:45.070839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:45.070839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:45.070839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:45.071624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 460/920] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:45.072861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d33f4e-acfc-4cbf-bedc-4db152ebf155 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.073182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d33f4e-acfc-4cbf-bedc-4db152ebf155 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.073478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d33f4e-acfc-4cbf-bedc-4db152ebf155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.091901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-02d33f4e-acfc-4cbf-bedc-4db152ebf155 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:45.093130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-02d33f4e-acfc-4cbf-bedc-4db152ebf155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:45.138111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2637dad8-1d2f-42b2-a07b-14c8409248d7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:45.162456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 461/921] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:45.163577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d33f4e-acfc-4cbf-bedc-4db152ebf155 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:45.174136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:45.188193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4bd0f-0251-422f-8785-73eea5aa8fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:45.188193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4bd0f-0251-422f-8785-73eea5aa8fad None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.188193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4bd0f-0251-422f-8785-73eea5aa8fad None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.188193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4bd0f-0251-422f-8785-73eea5aa8fad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.188880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4bd0f-0251-422f-8785-73eea5aa8fad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:45.190803 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4bd0f-0251-422f-8785-73eea5aa8fad None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:45.262631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 461/922] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:41:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:41:45.263534 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4bd0f-0251-422f-8785-73eea5aa8fad None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:45.286186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d33f4e-acfc-4cbf-bedc-4db152ebf155 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:45.333644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ebfb7-d115-413c-a34d-0ae0fa48fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:45.334181 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ebfb7-d115-413c-a34d-0ae0fa48fc5b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.334460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ebfb7-d115-413c-a34d-0ae0fa48fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.334729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ebfb7-d115-413c-a34d-0ae0fa48fc5b None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.335395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ebfb7-d115-413c-a34d-0ae0fa48fc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:45.335696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ebfb7-d115-413c-a34d-0ae0fa48fc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:45.337119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ebfb7-d115-413c-a34d-0ae0fa48fc5b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:45.354758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 462/923] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:45.354758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8bd13-6834-483a-8fa2-3bc0e49bccb1 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.354758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8bd13-6834-483a-8fa2-3bc0e49bccb1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.354758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8bd13-6834-483a-8fa2-3bc0e49bccb1 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.354758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 463/924] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:45.356838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8bd13-6834-483a-8fa2-3bc0e49bccb1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:45.375973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:45.399775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4bd0f-0251-422f-8785-73eea5aa8fad None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:45.423784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:45.439949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd192471-d12c-4548-8b96-84f3e023a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:45.439949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd192471-d12c-4548-8b96-84f3e023a428 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.440259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd192471-d12c-4548-8b96-84f3e023a428 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.440466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd192471-d12c-4548-8b96-84f3e023a428 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.441211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd192471-d12c-4548-8b96-84f3e023a428 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:45.443127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd192471-d12c-4548-8b96-84f3e023a428 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:45.467245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8bd13-6834-483a-8fa2-3bc0e49bccb1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:45.500200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939aaf2-940f-4f60-af3c-a72f2e67ed91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:45.500843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939aaf2-940f-4f60-af3c-a72f2e67ed91 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.501159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939aaf2-940f-4f60-af3c-a72f2e67ed91 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.501497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939aaf2-940f-4f60-af3c-a72f2e67ed91 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.508984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939aaf2-940f-4f60-af3c-a72f2e67ed91 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:45.513160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939aaf2-940f-4f60-af3c-a72f2e67ed91 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b939aaf2-940f-4f60-af3c-a72f2e67ed91 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:45.522124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:45.523136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:45.524148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:45.524148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:45.524148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:45.524148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:45.524148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:45.524148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:45.524148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:45.524148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:45.524148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:45.524148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:45.524148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 464/925] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Aug 11 09:41:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:45.524741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722bf04-1a4d-4dec-ae7f-ec8f823e3766 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.525234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722bf04-1a4d-4dec-ae7f-ec8f823e3766 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.525234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722bf04-1a4d-4dec-ae7f-ec8f823e3766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.541066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 462/926] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:41:45 2022] POST /identity/v3/services => generated 241 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:45.541066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd192471-d12c-4548-8b96-84f3e023a428 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:45.545878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1722bf04-1a4d-4dec-ae7f-ec8f823e3766 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:45.547383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1722bf04-1a4d-4dec-ae7f-ec8f823e3766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:45.633699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 465/927] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:45.661111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722bf04-1a4d-4dec-ae7f-ec8f823e3766 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:45.671551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd192471-d12c-4548-8b96-84f3e023a428 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:45.724978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:45.727820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9e725-eb59-4b69-b727-129740c6f4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:45.729721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9e725-eb59-4b69-b727-129740c6f4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.730430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9e725-eb59-4b69-b727-129740c6f4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.730430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9e725-eb59-4b69-b727-129740c6f4e8 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.731327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9e725-eb59-4b69-b727-129740c6f4e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:45.731564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9e725-eb59-4b69-b727-129740c6f4e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:45.733050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9e725-eb59-4b69-b727-129740c6f4e8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:45.755861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:45.757136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 463/928] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Aug 11 09:41:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:45.762351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9e725-eb59-4b69-b727-129740c6f4e8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:45.812608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722bf04-1a4d-4dec-ae7f-ec8f823e3766 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:45.858297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf80a6-a9c2-482a-a443-7bb4e32ac9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:45.859148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf80a6-a9c2-482a-a443-7bb4e32ac9ca None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.859430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf80a6-a9c2-482a-a443-7bb4e32ac9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.860002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf80a6-a9c2-482a-a443-7bb4e32ac9ca None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.860607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf80a6-a9c2-482a-a443-7bb4e32ac9ca None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:45.862109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf80a6-a9c2-482a-a443-7bb4e32ac9ca None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-3bcf80a6-a9c2-482a-a443-7bb4e32ac9ca None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:45.874971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:45.875934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 09:41:45.876784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:45.877553 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9e725-eb59-4b69-b727-129740c6f4e8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:45.877628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 466/929] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Aug 11 09:41:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:45.884290 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf80a6-a9c2-482a-a443-7bb4e32ac9ca None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:45.922440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844bca6-d48b-4209-8bbe-adfc15ef800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:45.923084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844bca6-d48b-4209-8bbe-adfc15ef800c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:45.923365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844bca6-d48b-4209-8bbe-adfc15ef800c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:45.923639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844bca6-d48b-4209-8bbe-adfc15ef800c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:45.935849 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844bca6-d48b-4209-8bbe-adfc15ef800c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:45.937438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844bca6-d48b-4209-8bbe-adfc15ef800c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e844bca6-d48b-4209-8bbe-adfc15ef800c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:45.953870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:45.954861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:45.955740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:45.955740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:45.955740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:45.955740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 464/930] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:45.960584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844bca6-d48b-4209-8bbe-adfc15ef800c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:46.019765 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf80a6-a9c2-482a-a443-7bb4e32ac9ca None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:46.025141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:46.031136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:46.063249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565fe8b-014f-4bcf-8092-779e286a6101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:46.064469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565fe8b-014f-4bcf-8092-779e286a6101 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:46.065275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565fe8b-014f-4bcf-8092-779e286a6101 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:46.065782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565fe8b-014f-4bcf-8092-779e286a6101 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:46.067036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565fe8b-014f-4bcf-8092-779e286a6101 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:46.067844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565fe8b-014f-4bcf-8092-779e286a6101 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:46.070008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565fe8b-014f-4bcf-8092-779e286a6101 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:46.082759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 467/931] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Aug 11 09:41:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:46.084173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565fe8b-014f-4bcf-8092-779e286a6101 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:46.087126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844bca6-d48b-4209-8bbe-adfc15ef800c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:46.124158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbc29f-63b3-4a51-9682-271c5d726221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:46.124774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbc29f-63b3-4a51-9682-271c5d726221 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:46.125084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbc29f-63b3-4a51-9682-271c5d726221 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:46.125359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbc29f-63b3-4a51-9682-271c5d726221 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:46.126012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbc29f-63b3-4a51-9682-271c5d726221 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:46.126318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbc29f-63b3-4a51-9682-271c5d726221 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:46.127752 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbc29f-63b3-4a51-9682-271c5d726221 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:46.141164 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 465/932] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:46.146680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbc29f-63b3-4a51-9682-271c5d726221 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:46.213271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565fe8b-014f-4bcf-8092-779e286a6101 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:46.252534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dcc4e-246f-4491-b04c-e9e7083165ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:46.253716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dcc4e-246f-4491-b04c-e9e7083165ab None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:46.253999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dcc4e-246f-4491-b04c-e9e7083165ab None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:46.254274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149dcc4e-246f-4491-b04c-e9e7083165ab None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:46.254916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149dcc4e-246f-4491-b04c-e9e7083165ab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:46.255217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149dcc4e-246f-4491-b04c-e9e7083165ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:46.256943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149dcc4e-246f-4491-b04c-e9e7083165ab None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:46.261918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 468/933] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Aug 11 09:41:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:46.267481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dcc4e-246f-4491-b04c-e9e7083165ab None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:46.270807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbc29f-63b3-4a51-9682-271c5d726221 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:46.288908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:46.311463 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:46.318733 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc4432-1670-4780-a5b6-a33ff61be846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:46.320148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc4432-1670-4780-a5b6-a33ff61be846 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:46.320710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc4432-1670-4780-a5b6-a33ff61be846 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:46.321194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc4432-1670-4780-a5b6-a33ff61be846 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:46.330726 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc4432-1670-4780-a5b6-a33ff61be846 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:46.333410 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc4432-1670-4780-a5b6-a33ff61be846 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-26fc4432-1670-4780-a5b6-a33ff61be846 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:46.348812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:46.349814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:46.350685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:46.350685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:46.350685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:46.350685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:46.350685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:46.350685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:46.350685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:46.350685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:46.355489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 466/934] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:46.357293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc4432-1670-4780-a5b6-a33ff61be846 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:46.384912 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149dcc4e-246f-4491-b04c-e9e7083165ab None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:46.423616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f8f1b-88fe-4b93-b892-860cb32a3223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:46.424131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f8f1b-88fe-4b93-b892-860cb32a3223 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:46.424718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f8f1b-88fe-4b93-b892-860cb32a3223 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:46.424718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f8f1b-88fe-4b93-b892-860cb32a3223 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:46.426079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f8f1b-88fe-4b93-b892-860cb32a3223 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:46.427456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f8f1b-88fe-4b93-b892-860cb32a3223 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:46.484933 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc4432-1670-4780-a5b6-a33ff61be846 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:46.516809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 469/935] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:41:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:41:46.530351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ebb97-71fe-4674-9309-b64c54d48cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:46.531001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ebb97-71fe-4674-9309-b64c54d48cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:46.531275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ebb97-71fe-4674-9309-b64c54d48cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:46.531547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ebb97-71fe-4674-9309-b64c54d48cf8 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:46.532279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ebb97-71fe-4674-9309-b64c54d48cf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:46.532587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ebb97-71fe-4674-9309-b64c54d48cf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:46.534815 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ebb97-71fe-4674-9309-b64c54d48cf8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:46.557181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 467/936] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:46.559279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f8f1b-88fe-4b93-b892-860cb32a3223 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:46.671146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:46.686941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f8f1b-88fe-4b93-b892-860cb32a3223 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:46.717403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20325c38-bc82-4be2-b985-1e181293788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:46.718363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20325c38-bc82-4be2-b985-1e181293788d None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:46.718815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20325c38-bc82-4be2-b985-1e181293788d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:46.720477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20325c38-bc82-4be2-b985-1e181293788d None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/813d8cff9d7d49d5995bfc272fee88f3/roles/1a335dba62e246ec820c70f60d2558b9` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:46.722614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20325c38-bc82-4be2-b985-1e181293788d None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=813d8cff9d7d49d5995bfc272fee88f3, role_id=1a335dba62e246ec820c70f60d2558b9)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:46.724387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20325c38-bc82-4be2-b985-1e181293788d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:46.800682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 470/937] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:41:46 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/813d8cff9d7d49d5995bfc272fee88f3/roles/1a335dba62e246ec820c70f60d2558b9 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:41:47.430590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f190f-7e61-42e4-bce4-faf6ad1f985b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:47.431275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f190f-7e61-42e4-bce4-faf6ad1f985b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:47.431789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f190f-7e61-42e4-bce4-faf6ad1f985b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:47.433734 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 468/938] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:47.439883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c3acf-aba8-46bb-badf-f11d1f8f668d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:47.441650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c3acf-aba8-46bb-badf-f11d1f8f668d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:47.442537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c3acf-aba8-46bb-badf-f11d1f8f668d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:47.462428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-858c3acf-aba8-46bb-badf-f11d1f8f668d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:47.463961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-858c3acf-aba8-46bb-badf-f11d1f8f668d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:47.536817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 471/939] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:47.560530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f190f-7e61-42e4-bce4-faf6ad1f985b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:47.591107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:47.670446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f190f-7e61-42e4-bce4-faf6ad1f985b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:47.679202 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaea131-3b3b-4583-8c76-2e7ee5e07fb9 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:47.679760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaea131-3b3b-4583-8c76-2e7ee5e07fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:47.680245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaea131-3b3b-4583-8c76-2e7ee5e07fb9 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:47.682061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 472/940] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:47.687193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd997f-70f9-4b00-80b6-1cf260a61f7d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:47.687470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd997f-70f9-4b00-80b6-1cf260a61f7d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:47.687741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd997f-70f9-4b00-80b6-1cf260a61f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:47.707366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-78bd997f-70f9-4b00-80b6-1cf260a61f7d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:47.708527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-78bd997f-70f9-4b00-80b6-1cf260a61f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:47.716667 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a54578-9296-4885-969e-a04c73cdecc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:47.717757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a54578-9296-4885-969e-a04c73cdecc0 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:47.719129 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a54578-9296-4885-969e-a04c73cdecc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:47.719712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a54578-9296-4885-969e-a04c73cdecc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:47.721212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a54578-9296-4885-969e-a04c73cdecc0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:47.724211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a54578-9296-4885-969e-a04c73cdecc0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:47.728288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: WARNING keystone.server.flask.common [None req-a7a54578-9296-4885-969e-a04c73cdecc0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 11 09:41:47.776945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.sql.core [None req-a7a54578-9296-4885-969e-a04c73cdecc0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 11 09:41:47.776945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 11 09:41:47.776945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [parameters: {'id': '46e46702dd574f2783ed782a2cee9efe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 11 09:41:47.776945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81638) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a7a54578-9296-4885-969e-a04c73cdecc0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 46e46702dd574f2783ed782a2cee9efe.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 46e46702dd574f2783ed782a2cee9efe. Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application conn.query(q) Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 09:41:47.787136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result.read() Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 11 09:41:47.788003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application next(self.gen) Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application yield resource Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application next(self.gen) Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.session.rollback() Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.force_reraise() Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise self.value Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application session.commit() Aug 11 09:41:47.788967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._prepare_impl() Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.session.flush() Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._flush(objects) Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application compat.raise_( Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application flush_context.execute() Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 11 09:41:47.790007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rec.execute(self) Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 11 09:41:47.790963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application conn.query(q) Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result.read() Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 09:41:47.792192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [parameters: {'id': '46e46702dd574f2783ed782a2cee9efe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:47.794101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:47.795842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:47.797870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:47.797870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 11 09:41:47.797870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 11 09:41:47.797870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:47.797870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:47.797870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 11 09:41:47.797870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 11 09:41:47.797870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 46e46702dd574f2783ed782a2cee9efe. Aug 11 09:41:47.797870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:47.800017 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 473/941] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:47.800974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 469/942] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Aug 11 09:41:47 2022] POST /identity/v3/projects => generated 262 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:47.807210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd997f-70f9-4b00-80b6-1cf260a61f7d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:47.811995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a54578-9296-4885-969e-a04c73cdecc0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:47.831708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:47.905762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:47.922301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a54578-9296-4885-969e-a04c73cdecc0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:47.938260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd997f-70f9-4b00-80b6-1cf260a61f7d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:47.960097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f586c95-8376-43d5-8496-5e9a63ce95f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:47.960661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f586c95-8376-43d5-8496-5e9a63ce95f5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:47.960951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f586c95-8376-43d5-8496-5e9a63ce95f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:47.961258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f586c95-8376-43d5-8496-5e9a63ce95f5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:47.961856 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f586c95-8376-43d5-8496-5e9a63ce95f5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:47.963442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f586c95-8376-43d5-8496-5e9a63ce95f5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-9f586c95-8376-43d5-8496-5e9a63ce95f5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:47.972507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:47.974388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:47.975254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 470/943] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Aug 11 09:41:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:47.982191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f586c95-8376-43d5-8496-5e9a63ce95f5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:47.983577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb2627-1556-4840-89b8-cc1a76aedf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:47.984407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb2627-1556-4840-89b8-cc1a76aedf3b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:47.986093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb2627-1556-4840-89b8-cc1a76aedf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:47.986093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb2627-1556-4840-89b8-cc1a76aedf3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:47.993194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb2627-1556-4840-89b8-cc1a76aedf3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:47.994906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb2627-1556-4840-89b8-cc1a76aedf3b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-c5fb2627-1556-4840-89b8-cc1a76aedf3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:48.007689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:48.008617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:48.009557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:48.009557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:48.009557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:48.009557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:48.009557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:48.009557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:48.009557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:48.009557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:48.011604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 474/944] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:48.017365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb2627-1556-4840-89b8-cc1a76aedf3b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:48.099886 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:48.111527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f586c95-8376-43d5-8496-5e9a63ce95f5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:48.126998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb2627-1556-4840-89b8-cc1a76aedf3b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:48.154911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb02ad8-7800-4331-9f2b-581786a0de68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:48.155627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb02ad8-7800-4331-9f2b-581786a0de68 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:48.155912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb02ad8-7800-4331-9f2b-581786a0de68 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:48.156187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb02ad8-7800-4331-9f2b-581786a0de68 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:48.156816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb02ad8-7800-4331-9f2b-581786a0de68 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:48.157145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb02ad8-7800-4331-9f2b-581786a0de68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:48.159116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb02ad8-7800-4331-9f2b-581786a0de68 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:48.168199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbf7f2-7b02-476f-ac82-7c1f02e46973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:48.169239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbf7f2-7b02-476f-ac82-7c1f02e46973 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:48.169728 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbf7f2-7b02-476f-ac82-7c1f02e46973 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:48.170203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fbf7f2-7b02-476f-ac82-7c1f02e46973 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:48.171093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbf7f2-7b02-476f-ac82-7c1f02e46973 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:48.171626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbf7f2-7b02-476f-ac82-7c1f02e46973 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:48.173371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fbf7f2-7b02-476f-ac82-7c1f02e46973 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:48.173619 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 471/945] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Aug 11 09:41:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:48.174754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb02ad8-7800-4331-9f2b-581786a0de68 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:48.194982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:48.214863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 475/946] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:48 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:48.295384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb02ad8-7800-4331-9f2b-581786a0de68 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:48.335049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d599f1b-c428-48b0-b810-8969d788c5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:48.335676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d599f1b-c428-48b0-b810-8969d788c5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:48.335953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d599f1b-c428-48b0-b810-8969d788c5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:48.336272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d599f1b-c428-48b0-b810-8969d788c5d6 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:48.336918 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d599f1b-c428-48b0-b810-8969d788c5d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:48.337244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d599f1b-c428-48b0-b810-8969d788c5d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:48.338684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d599f1b-c428-48b0-b810-8969d788c5d6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:48.349161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 472/947] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Aug 11 09:41:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:48.354140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbf7f2-7b02-476f-ac82-7c1f02e46973 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:48.470693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fbf7f2-7b02-476f-ac82-7c1f02e46973 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:48.512650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bdf1c-63b2-4815-b43e-20f145657e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:48.513639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bdf1c-63b2-4815-b43e-20f145657e44 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:48.514170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bdf1c-63b2-4815-b43e-20f145657e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:48.514653 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bdf1c-63b2-4815-b43e-20f145657e44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:48.515559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bdf1c-63b2-4815-b43e-20f145657e44 None admin] RBAC: Adding query filter params (interface=public, service_id=61bbb8ceb36d45db9e56ef342c361de8, region_id=RegionOne) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:48.516074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bdf1c-63b2-4815-b43e-20f145657e44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:48.517711 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bdf1c-63b2-4815-b43e-20f145657e44 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:48.520679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:48.525378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 476/948] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Aug 11 09:41:48 2022] GET /identity/v3/endpoints?service_id=61bbb8ceb36d45db9e56ef342c361de8&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:48.531004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d599f1b-c428-48b0-b810-8969d788c5d6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:48.617482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:48.625506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a19ea-59b3-4bd4-b605-e279f9bf6b22 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:48.625654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a19ea-59b3-4bd4-b605-e279f9bf6b22 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:48.625907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a19ea-59b3-4bd4-b605-e279f9bf6b22 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:48.627669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 477/949] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:48.632362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f5760-4633-4111-a788-76d78888b941 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:48.632677 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f5760-4633-4111-a788-76d78888b941 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:48.633003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f5760-4633-4111-a788-76d78888b941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:48.647414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d599f1b-c428-48b0-b810-8969d788c5d6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:48.651291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9c3f5760-4633-4111-a788-76d78888b941 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:48.652596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9c3f5760-4633-4111-a788-76d78888b941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:48.685689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67347ee-c9d7-405e-80bc-104a91686aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:48.686469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67347ee-c9d7-405e-80bc-104a91686aaf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:48.686853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67347ee-c9d7-405e-80bc-104a91686aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:48.687228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67347ee-c9d7-405e-80bc-104a91686aaf None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:48.687930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67347ee-c9d7-405e-80bc-104a91686aaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:48.689400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67347ee-c9d7-405e-80bc-104a91686aaf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:48.697240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 473/950] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Aug 11 09:41:48 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 09:41:48.736343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 478/951] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:48.758756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67347ee-c9d7-405e-80bc-104a91686aaf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:48.873897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09c088-41f4-4251-8866-bdc55d7e2437 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:48.874233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09c088-41f4-4251-8866-bdc55d7e2437 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:48.874491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09c088-41f4-4251-8866-bdc55d7e2437 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:48.875833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 479/952] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:48.879486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67347ee-c9d7-405e-80bc-104a91686aaf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:48.881476 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1e524-ce80-46dc-8147-1ae5bbaa55a8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:48.881987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1e524-ce80-46dc-8147-1ae5bbaa55a8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:48.882457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1e524-ce80-46dc-8147-1ae5bbaa55a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:48.905068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0fd1e524-ce80-46dc-8147-1ae5bbaa55a8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:48.905068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0fd1e524-ce80-46dc-8147-1ae5bbaa55a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:48.919622 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df070c-7179-457a-95c1-045c0ceba733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:48.920310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df070c-7179-457a-95c1-045c0ceba733 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:48.920661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df070c-7179-457a-95c1-045c0ceba733 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:48.920938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df070c-7179-457a-95c1-045c0ceba733 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:48.921573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df070c-7179-457a-95c1-045c0ceba733 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:48.923045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df070c-7179-457a-95c1-045c0ceba733 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-95df070c-7179-457a-95c1-045c0ceba733 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:48.934688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:48.935758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:48.936587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 474/953] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:48.944115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df070c-7179-457a-95c1-045c0ceba733 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:48.948792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:48.990766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 480/954] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:49.014469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1e524-ce80-46dc-8147-1ae5bbaa55a8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:49.057476 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df070c-7179-457a-95c1-045c0ceba733 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:49.087476 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:49.096928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bf73d-a1da-4af9-b0fe-a2b73351eee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:49.097837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bf73d-a1da-4af9-b0fe-a2b73351eee6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:49.104055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bf73d-a1da-4af9-b0fe-a2b73351eee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:49.104055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bf73d-a1da-4af9-b0fe-a2b73351eee6 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:49.104055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bf73d-a1da-4af9-b0fe-a2b73351eee6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:49.104055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bf73d-a1da-4af9-b0fe-a2b73351eee6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:49.104055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bf73d-a1da-4af9-b0fe-a2b73351eee6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:49.106717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 475/955] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:49.117059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bf73d-a1da-4af9-b0fe-a2b73351eee6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:49.139184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1e524-ce80-46dc-8147-1ae5bbaa55a8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:49.184627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab77ed-4123-4388-b513-f71152c1a9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:49.185383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab77ed-4123-4388-b513-f71152c1a9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:49.185813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab77ed-4123-4388-b513-f71152c1a9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:49.186434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab77ed-4123-4388-b513-f71152c1a9a5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:49.201005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:49.201883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab77ed-4123-4388-b513-f71152c1a9a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b9ab77ed-4123-4388-b513-f71152c1a9a5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:41:49.208668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:49.213109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:49.213995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 481/956] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Aug 11 09:41:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:49.216258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab77ed-4123-4388-b513-f71152c1a9a5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:49.247586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bf73d-a1da-4af9-b0fe-a2b73351eee6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:49.284689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d23a7-a80c-4d2a-b578-174d6c9338e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:49.285710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d23a7-a80c-4d2a-b578-174d6c9338e6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:49.286212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d23a7-a80c-4d2a-b578-174d6c9338e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:49.286684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d23a7-a80c-4d2a-b578-174d6c9338e6 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:49.287530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d23a7-a80c-4d2a-b578-174d6c9338e6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:49.288085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d23a7-a80c-4d2a-b578-174d6c9338e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:49.289724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d23a7-a80c-4d2a-b578-174d6c9338e6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:49.294982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 476/957] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:49 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:49.337324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab77ed-4123-4388-b513-f71152c1a9a5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:49.357309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:49.377450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b58ca-0eda-4f35-8ec3-c4c4878d3e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:49.380015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b58ca-0eda-4f35-8ec3-c4c4878d3e60 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:49.380730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b58ca-0eda-4f35-8ec3-c4c4878d3e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:49.381318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b58ca-0eda-4f35-8ec3-c4c4878d3e60 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:49.382335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b58ca-0eda-4f35-8ec3-c4c4878d3e60 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:49.382854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b58ca-0eda-4f35-8ec3-c4c4878d3e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:49.390573 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b58ca-0eda-4f35-8ec3-c4c4878d3e60 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:49.395459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 482/958] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Aug 11 09:41:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:49.402426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d23a7-a80c-4d2a-b578-174d6c9338e6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:49.513942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d23a7-a80c-4d2a-b578-174d6c9338e6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:49.545312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:49.551244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eae75d-3ddd-4915-85a9-e01ece9e8758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:49.551903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eae75d-3ddd-4915-85a9-e01ece9e8758 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:49.552181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eae75d-3ddd-4915-85a9-e01ece9e8758 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:49.552454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eae75d-3ddd-4915-85a9-e01ece9e8758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:49.563994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eae75d-3ddd-4915-85a9-e01ece9e8758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:49.565495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eae75d-3ddd-4915-85a9-e01ece9e8758 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-44eae75d-3ddd-4915-85a9-e01ece9e8758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:49.580501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:49.581605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:49.582513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:49.582513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:49.582513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:49.583167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 477/959] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:49.590106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b58ca-0eda-4f35-8ec3-c4c4878d3e60 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:49.701775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b58ca-0eda-4f35-8ec3-c4c4878d3e60 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:49.736483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695b442-852b-4680-a563-69c1807092cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:49.737670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695b442-852b-4680-a563-69c1807092cd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:49.737956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695b442-852b-4680-a563-69c1807092cd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:49.738236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695b442-852b-4680-a563-69c1807092cd None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:49.738860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695b442-852b-4680-a563-69c1807092cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:49.739162 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695b442-852b-4680-a563-69c1807092cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:49.740523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695b442-852b-4680-a563-69c1807092cd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:49.752656 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 483/960] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:49.758612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eae75d-3ddd-4915-85a9-e01ece9e8758 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:49.870664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eae75d-3ddd-4915-85a9-e01ece9e8758 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:49.908068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5f22e-9f0e-492f-a522-0a6f6448c162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:49.908721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5f22e-9f0e-492f-a522-0a6f6448c162 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:49.909606 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5f22e-9f0e-492f-a522-0a6f6448c162 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:49.909879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5f22e-9f0e-492f-a522-0a6f6448c162 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:49.920727 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5f22e-9f0e-492f-a522-0a6f6448c162 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:49.922203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5f22e-9f0e-492f-a522-0a6f6448c162 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-cea5f22e-9f0e-492f-a522-0a6f6448c162 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:49.935386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:49.936326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:49.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:49.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:49.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:49.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:49.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:49.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:49.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:49.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:49.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 09:41:49.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:49.937282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 478/961] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Aug 11 09:41:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:49.944329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695b442-852b-4680-a563-69c1807092cd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:49.993544 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:50.053736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695b442-852b-4680-a563-69c1807092cd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:50.089530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528896e-4d7c-4308-9cdf-45348333c8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:50.090139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528896e-4d7c-4308-9cdf-45348333c8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:50.090426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528896e-4d7c-4308-9cdf-45348333c8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:50.090704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528896e-4d7c-4308-9cdf-45348333c8e6 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:50.091436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528896e-4d7c-4308-9cdf-45348333c8e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:50.091744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528896e-4d7c-4308-9cdf-45348333c8e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:50.093201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528896e-4d7c-4308-9cdf-45348333c8e6 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:50.111518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 484/962] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Aug 11 09:41:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:50.117781 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5f22e-9f0e-492f-a522-0a6f6448c162 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:50.166285 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:50.224445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5f22e-9f0e-492f-a522-0a6f6448c162 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:50.260132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dece80d-57f1-4a4e-b854-9ae5e856033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:50.260742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dece80d-57f1-4a4e-b854-9ae5e856033e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:50.261037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dece80d-57f1-4a4e-b854-9ae5e856033e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:50.261374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dece80d-57f1-4a4e-b854-9ae5e856033e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:50.335916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dece80d-57f1-4a4e-b854-9ae5e856033e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:50.337385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dece80d-57f1-4a4e-b854-9ae5e856033e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:50.384607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:50.442722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 479/963] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:41:50 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:41:50.448236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528896e-4d7c-4308-9cdf-45348333c8e6 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:50.458028 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c866f6-f053-4090-9847-061d1ce086b2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:50.458326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c866f6-f053-4090-9847-061d1ce086b2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:50.458694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c866f6-f053-4090-9847-061d1ce086b2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:50.460282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 480/964] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:50.467154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb61f5-55b2-430b-a734-f8af0dd36193 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:50.467432 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb61f5-55b2-430b-a734-f8af0dd36193 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:50.467705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb61f5-55b2-430b-a734-f8af0dd36193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:50.487097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4ccb61f5-55b2-430b-a734-f8af0dd36193 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:50.488720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4ccb61f5-55b2-430b-a734-f8af0dd36193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:50.563097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:50.568342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 481/965] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:50.568492 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528896e-4d7c-4308-9cdf-45348333c8e6 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:50.592376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb61f5-55b2-430b-a734-f8af0dd36193 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:50.605639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ed1a3-b33e-45dc-9577-c6588566b06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:50.606292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ed1a3-b33e-45dc-9577-c6588566b06a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:50.606751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ed1a3-b33e-45dc-9577-c6588566b06a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:50.607355 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ed1a3-b33e-45dc-9577-c6588566b06a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:50.619275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ed1a3-b33e-45dc-9577-c6588566b06a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:50.620809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ed1a3-b33e-45dc-9577-c6588566b06a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-0b9ed1a3-b33e-45dc-9577-c6588566b06a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:41:50.637651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:50.639752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:41:50.641035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:41:50.641035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:41:50.641035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:50.641035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 485/966] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:50.645395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ed1a3-b33e-45dc-9577-c6588566b06a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:50.677952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:50.711549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb61f5-55b2-430b-a734-f8af0dd36193 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:50.747665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b520e-0817-4922-b680-7319bf772ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:50.748275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b520e-0817-4922-b680-7319bf772ae9 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:50.748692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b520e-0817-4922-b680-7319bf772ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:50.748970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b520e-0817-4922-b680-7319bf772ae9 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:50.749742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b520e-0817-4922-b680-7319bf772ae9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:50.751159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b520e-0817-4922-b680-7319bf772ae9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:50.755845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ed1a3-b33e-45dc-9577-c6588566b06a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:50.789921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b2d0e-c50a-4cb8-b57d-185ab1e86ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:50.790523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b2d0e-c50a-4cb8-b57d-185ab1e86ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:50.790802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b2d0e-c50a-4cb8-b57d-185ab1e86ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:50.791073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b2d0e-c50a-4cb8-b57d-185ab1e86ed3 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:50.791896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b2d0e-c50a-4cb8-b57d-185ab1e86ed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:50.792239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b2d0e-c50a-4cb8-b57d-185ab1e86ed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:50.793641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b2d0e-c50a-4cb8-b57d-185ab1e86ed3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:50.794121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.sql.core [None req-7f0b520e-0817-4922-b680-7319bf772ae9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 11 09:41:50.794121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 11 09:41:50.794121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [parameters: {'user_id': '72786b48720449fd8197c45ecfc9cd20', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 11 09:41:50.794121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81638) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-7f0b520e-0817-4922-b680-7319bf772ae9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application conn.query(q) Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result.read() Aug 11 09:41:50.805467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 09:41:50.806333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application next(self.gen) Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application yield resource Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application next(self.gen) Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.session.rollback() Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.force_reraise() Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise self.value Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application session.commit() Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 11 09:41:50.807248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._prepare_impl() Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.session.flush() Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._flush(objects) Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application compat.raise_( Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application flush_context.execute() Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rec.execute(self) Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 11 09:41:50.808141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 09:41:50.813876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application conn.query(q) Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result.read() Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 09:41:50.816495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [parameters: {'user_id': '72786b48720449fd8197c45ecfc9cd20', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:50.819077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 11 09:41:50.821048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 11 09:41:50.823247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 11 09:41:50.823247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 11 09:41:50.823247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 11 09:41:50.823247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 11 09:41:50.823247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 11 09:41:50.823247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:50.823973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 486/967] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:50.823973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b2d0e-c50a-4cb8-b57d-185ab1e86ed3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:50.824457 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 482/968] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Aug 11 09:41:50 2022] POST /identity/v3/users => generated 160 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:50.824457 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b520e-0817-4922-b680-7319bf772ae9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:50.834900 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:50.928070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b2d0e-c50a-4cb8-b57d-185ab1e86ed3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:50.936976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b520e-0817-4922-b680-7319bf772ae9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:50.963355 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6af55-11fb-4f83-9fe2-bf1158ac6321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:50.963907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6af55-11fb-4f83-9fe2-bf1158ac6321 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:50.964208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6af55-11fb-4f83-9fe2-bf1158ac6321 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:50.964454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6af55-11fb-4f83-9fe2-bf1158ac6321 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:50.972667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6af55-11fb-4f83-9fe2-bf1158ac6321 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:50.974133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6af55-11fb-4f83-9fe2-bf1158ac6321 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:50.976250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42325938-c017-46a2-b2c3-8f4aa138f0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:50.977208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42325938-c017-46a2-b2c3-8f4aa138f0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:50.977668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42325938-c017-46a2-b2c3-8f4aa138f0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:50.978216 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42325938-c017-46a2-b2c3-8f4aa138f0f1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:50.981462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-31d6af55-11fb-4f83-9fe2-bf1158ac6321 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:41:50.985351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:50.986220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:50.987726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:41:50.987726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:41:50.987726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:41:50.987726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:41:50.987726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:41:50.987726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:41:50.987726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:41:50.987726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:50.987726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 487/969] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:50.988849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf54ab-2755-49d4-9bda-dd973eb43067 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:50.989077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf54ab-2755-49d4-9bda-dd973eb43067 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:50.989359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf54ab-2755-49d4-9bda-dd973eb43067 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:50.990713 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 488/970] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:50.993271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42325938-c017-46a2-b2c3-8f4aa138f0f1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:50.995016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42325938-c017-46a2-b2c3-8f4aa138f0f1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:50.995374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf54ab-2755-49d4-9bda-dd973eb43067 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-42325938-c017-46a2-b2c3-8f4aa138f0f1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:41:51.007912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:51.009132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:51.010399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:51.010399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:51.010399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:41:51.010399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:41:51.010399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:41:51.010399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:41:51.010399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:41:51.010399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:41:51.010399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 09:41:51.010399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:51.013688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 483/971] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:41:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:51.015372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51a362-9dd1-4281-962f-0fbd679426c7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:51.015897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51a362-9dd1-4281-962f-0fbd679426c7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:51.016368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51a362-9dd1-4281-962f-0fbd679426c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:51.036070 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8e51a362-9dd1-4281-962f-0fbd679426c7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:51.037930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8e51a362-9dd1-4281-962f-0fbd679426c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:51.072688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:51.107967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf54ab-2755-49d4-9bda-dd973eb43067 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:51.121382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 484/972] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:51.123007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51a362-9dd1-4281-962f-0fbd679426c7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:51.145056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f604001-5c32-4cf6-9395-f3bfd3437d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:51.145908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f604001-5c32-4cf6-9395-f3bfd3437d4a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:51.146188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f604001-5c32-4cf6-9395-f3bfd3437d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:51.146616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f604001-5c32-4cf6-9395-f3bfd3437d4a None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:51.147139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f604001-5c32-4cf6-9395-f3bfd3437d4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:51.147438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f604001-5c32-4cf6-9395-f3bfd3437d4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:51.148850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f604001-5c32-4cf6-9395-f3bfd3437d4a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:51.161061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 489/973] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:41:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:51.161707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f604001-5c32-4cf6-9395-f3bfd3437d4a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:51.250116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51a362-9dd1-4281-962f-0fbd679426c7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:51.270428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f604001-5c32-4cf6-9395-f3bfd3437d4a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:51.279266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:51.289185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8495532-e328-4bf6-b3c2-9fa76353b5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:51.289824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8495532-e328-4bf6-b3c2-9fa76353b5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:51.295935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8495532-e328-4bf6-b3c2-9fa76353b5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:51.295935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8495532-e328-4bf6-b3c2-9fa76353b5d8 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:51.295935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8495532-e328-4bf6-b3c2-9fa76353b5d8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:51.295935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8495532-e328-4bf6-b3c2-9fa76353b5d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:51.295935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8495532-e328-4bf6-b3c2-9fa76353b5d8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:51.301966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:51.307611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd6861-e2f4-4564-9d20-08ea175ef56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:51.308313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd6861-e2f4-4564-9d20-08ea175ef56b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:51.308659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd6861-e2f4-4564-9d20-08ea175ef56b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:51.308998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd6861-e2f4-4564-9d20-08ea175ef56b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:51.309672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd6861-e2f4-4564-9d20-08ea175ef56b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:51.311118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd6861-e2f4-4564-9d20-08ea175ef56b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-48dd6861-e2f4-4564-9d20-08ea175ef56b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:51.321731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:51.329247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:51.330237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 490/974] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Aug 11 09:41:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:51.331289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd6861-e2f4-4564-9d20-08ea175ef56b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:51.349058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 485/975] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Aug 11 09:41:51 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:51.350124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8495532-e328-4bf6-b3c2-9fa76353b5d8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:51.446709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd6861-e2f4-4564-9d20-08ea175ef56b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:51.484865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1d486-db3e-4bc9-aff8-ff88a1393b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:51.485612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1d486-db3e-4bc9-aff8-ff88a1393b18 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:51.485886 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1d486-db3e-4bc9-aff8-ff88a1393b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:51.486155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1d486-db3e-4bc9-aff8-ff88a1393b18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:51.487434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1d486-db3e-4bc9-aff8-ff88a1393b18 None admin] RBAC: Adding query filter params (role.id=1a335dba62e246ec820c70f60d2558b9, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=813d8cff9d7d49d5995bfc272fee88f3) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:51.487737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1d486-db3e-4bc9-aff8-ff88a1393b18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:51.489350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1d486-db3e-4bc9-aff8-ff88a1393b18 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:51.495360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8495532-e328-4bf6-b3c2-9fa76353b5d8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:51.505128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 491/976] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:41:51 2022] GET /identity/v3/role_assignments?user.id=813d8cff9d7d49d5995bfc272fee88f3&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=1a335dba62e246ec820c70f60d2558b9 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:51.505128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e211d41-7743-44a3-a070-4573255383f5 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:51.505128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e211d41-7743-44a3-a070-4573255383f5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:51.505128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e211d41-7743-44a3-a070-4573255383f5 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:51.505128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 492/977] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:51.509058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64741c-ce4a-46d3-90c5-1c6d4482485a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:51.509058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64741c-ce4a-46d3-90c5-1c6d4482485a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:51.509058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64741c-ce4a-46d3-90c5-1c6d4482485a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:51.528996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ea64741c-ce4a-46d3-90c5-1c6d4482485a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:51.530275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ea64741c-ce4a-46d3-90c5-1c6d4482485a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:51.550888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f0002-9684-40f2-bfd9-726c321e724c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:51.550888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f0002-9684-40f2-bfd9-726c321e724c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:51.550888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f0002-9684-40f2-bfd9-726c321e724c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:51.551260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f0002-9684-40f2-bfd9-726c321e724c None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:51.551778 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f0002-9684-40f2-bfd9-726c321e724c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:51.552082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f0002-9684-40f2-bfd9-726c321e724c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:51.554064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f0002-9684-40f2-bfd9-726c321e724c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:51.561529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 486/978] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Aug 11 09:41:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:51.569223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f0002-9684-40f2-bfd9-726c321e724c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:51.628521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:51.646394 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:51.649569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 493/979] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:51.675087 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64741c-ce4a-46d3-90c5-1c6d4482485a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:51.702671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f0002-9684-40f2-bfd9-726c321e724c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:51.742343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb617b9b-1cfb-4d43-bb09-e11570f6ecc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:51.743221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb617b9b-1cfb-4d43-bb09-e11570f6ecc9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:51.743502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb617b9b-1cfb-4d43-bb09-e11570f6ecc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:51.743775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb617b9b-1cfb-4d43-bb09-e11570f6ecc9 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:51.744409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb617b9b-1cfb-4d43-bb09-e11570f6ecc9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:51.744710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb617b9b-1cfb-4d43-bb09-e11570f6ecc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:51.746090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb617b9b-1cfb-4d43-bb09-e11570f6ecc9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:51.756562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 487/980] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Aug 11 09:41:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:51.757756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb617b9b-1cfb-4d43-bb09-e11570f6ecc9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:51.782648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64741c-ce4a-46d3-90c5-1c6d4482485a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:51.818784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66012e58-fdb6-4b57-97e3-eb1c57e2d8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:51.819423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66012e58-fdb6-4b57-97e3-eb1c57e2d8a8 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:51.819706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66012e58-fdb6-4b57-97e3-eb1c57e2d8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:51.819982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66012e58-fdb6-4b57-97e3-eb1c57e2d8a8 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:51.820630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66012e58-fdb6-4b57-97e3-eb1c57e2d8a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:51.822043 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66012e58-fdb6-4b57-97e3-eb1c57e2d8a8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:51.863838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb617b9b-1cfb-4d43-bb09-e11570f6ecc9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:51.876507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:51.892092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 494/981] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:41:51 2022] POST /identity/v3/services => generated 242 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:41:51.905756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd591f8c-4eb7-4f83-9ee8-b6bb163637ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:51.906389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd591f8c-4eb7-4f83-9ee8-b6bb163637ce None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:51.906682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd591f8c-4eb7-4f83-9ee8-b6bb163637ce None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:51.906970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd591f8c-4eb7-4f83-9ee8-b6bb163637ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:51.907640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd591f8c-4eb7-4f83-9ee8-b6bb163637ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:51.910477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd591f8c-4eb7-4f83-9ee8-b6bb163637ce None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:51.990221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 488/982] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:41:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:53.588096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc74539-e60e-43c5-8dfc-b4363a09a0a8 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:53.588717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc74539-e60e-43c5-8dfc-b4363a09a0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:53.588717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc74539-e60e-43c5-8dfc-b4363a09a0a8 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:53.590136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 495/983] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:53.595337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7be4c5-3341-41be-82d6-438a18333846 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:53.595738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7be4c5-3341-41be-82d6-438a18333846 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:53.595901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7be4c5-3341-41be-82d6-438a18333846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:53.614776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9c7be4c5-3341-41be-82d6-438a18333846 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:53.615966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9c7be4c5-3341-41be-82d6-438a18333846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:53.692829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 489/984] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:53.714134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc74539-e60e-43c5-8dfc-b4363a09a0a8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:53.765288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:53.817802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc74539-e60e-43c5-8dfc-b4363a09a0a8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:53.852268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f35d88-66d4-42c9-85bf-a610223a1fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:53.853001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f35d88-66d4-42c9-85bf-a610223a1fad None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:53.853309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f35d88-66d4-42c9-85bf-a610223a1fad None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:53.853581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f35d88-66d4-42c9-85bf-a610223a1fad None admin] PATH_INFO: `/v3/services/image` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:53.854168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f35d88-66d4-42c9-85bf-a610223a1fad None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:53.855528 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f35d88-66d4-42c9-85bf-a610223a1fad None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b5f35d88-66d4-42c9-85bf-a610223a1fad None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:53.864119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:53.865409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 09:41:53.866305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:53.867632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 496/985] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Aug 11 09:41:53 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:53.871300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7be4c5-3341-41be-82d6-438a18333846 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:53.927420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d03670-39c8-41d4-9005-3667150b8f3e None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:53.927685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d03670-39c8-41d4-9005-3667150b8f3e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:53.927960 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d03670-39c8-41d4-9005-3667150b8f3e None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:53.929316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 497/986] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:53.934160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39693b55-cba5-496d-a47c-265898fb3c89 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:53.934438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39693b55-cba5-496d-a47c-265898fb3c89 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:53.935232 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39693b55-cba5-496d-a47c-265898fb3c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:53.954969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-39693b55-cba5-496d-a47c-265898fb3c89 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:53.956808 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-39693b55-cba5-496d-a47c-265898fb3c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:53.968688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:53.977403 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7be4c5-3341-41be-82d6-438a18333846 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.012947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce081b-406f-4889-9086-194973807ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.013649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce081b-406f-4889-9086-194973807ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.013934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce081b-406f-4889-9086-194973807ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.014199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce081b-406f-4889-9086-194973807ca8 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.014828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce081b-406f-4889-9086-194973807ca8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:54.015135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce081b-406f-4889-9086-194973807ca8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.016483 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce081b-406f-4889-9086-194973807ca8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.025053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 490/987] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:54.027545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce081b-406f-4889-9086-194973807ca8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:54.037054 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 498/988] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:54.037750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf97bb-5341-44c9-89df-392083f46cef None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.038181 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf97bb-5341-44c9-89df-392083f46cef None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.038482 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf97bb-5341-44c9-89df-392083f46cef None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.039941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 499/989] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:54.044825 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994d41-f8ee-443d-b0fc-c8cff14d7fae None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.045146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994d41-f8ee-443d-b0fc-c8cff14d7fae None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.045427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994d41-f8ee-443d-b0fc-c8cff14d7fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.063462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e4994d41-f8ee-443d-b0fc-c8cff14d7fae None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:54.064668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e4994d41-f8ee-443d-b0fc-c8cff14d7fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:54.100892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:54.128765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce081b-406f-4889-9086-194973807ca8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.132471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 500/990] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:41:54.133323 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4994d41-f8ee-443d-b0fc-c8cff14d7fae None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:54.163451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23306c-68ee-4218-ba25-2de3190ac8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.164063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23306c-68ee-4218-ba25-2de3190ac8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.164343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23306c-68ee-4218-ba25-2de3190ac8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.164628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23306c-68ee-4218-ba25-2de3190ac8a5 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.165287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23306c-68ee-4218-ba25-2de3190ac8a5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:54.165597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23306c-68ee-4218-ba25-2de3190ac8a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.166942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23306c-68ee-4218-ba25-2de3190ac8a5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.179787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 491/991] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:54 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:54.179787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23306c-68ee-4218-ba25-2de3190ac8a5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:54.200890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:54.254576 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4994d41-f8ee-443d-b0fc-c8cff14d7fae None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.293892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61fc043-5ff1-41ca-a7b5-19e8632966de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.294536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fc043-5ff1-41ca-a7b5-19e8632966de None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.294816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fc043-5ff1-41ca-a7b5-19e8632966de None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.295075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23306c-68ee-4218-ba25-2de3190ac8a5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.295204 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fc043-5ff1-41ca-a7b5-19e8632966de None admin] PATH_INFO: `/v3/services/compute` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.295684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61fc043-5ff1-41ca-a7b5-19e8632966de None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.297063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61fc043-5ff1-41ca-a7b5-19e8632966de None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-d61fc043-5ff1-41ca-a7b5-19e8632966de None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:54.306988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:54.308045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:54.308866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 501/992] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:54.314345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61fc043-5ff1-41ca-a7b5-19e8632966de None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:54.340900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f8899-9802-458a-a541-ee07a3f345d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.341577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f8899-9802-458a-a541-ee07a3f345d0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.341857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f8899-9802-458a-a541-ee07a3f345d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.342136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f8899-9802-458a-a541-ee07a3f345d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.342728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f8899-9802-458a-a541-ee07a3f345d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.344082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f8899-9802-458a-a541-ee07a3f345d0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2b6f8899-9802-458a-a541-ee07a3f345d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:54.353644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:54.354662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:54.355526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 492/993] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Aug 11 09:41:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:54.358261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:54.360972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f8899-9802-458a-a541-ee07a3f345d0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:54.428734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61fc043-5ff1-41ca-a7b5-19e8632966de None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.451351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:54.464662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e472b-2054-406e-9dc6-b088582d2503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.465511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e472b-2054-406e-9dc6-b088582d2503 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.470537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e472b-2054-406e-9dc6-b088582d2503 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.470537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e472b-2054-406e-9dc6-b088582d2503 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.470537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e472b-2054-406e-9dc6-b088582d2503 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:54.470537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e472b-2054-406e-9dc6-b088582d2503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.470537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e472b-2054-406e-9dc6-b088582d2503 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.470920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f8899-9802-458a-a541-ee07a3f345d0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.478184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 502/994] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:54.479299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e472b-2054-406e-9dc6-b088582d2503 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:54.508426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856694fb-8cbc-4efe-bf5c-e3d9654da8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.509052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856694fb-8cbc-4efe-bf5c-e3d9654da8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.509345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856694fb-8cbc-4efe-bf5c-e3d9654da8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.509625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856694fb-8cbc-4efe-bf5c-e3d9654da8e4 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.510257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856694fb-8cbc-4efe-bf5c-e3d9654da8e4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:54.510562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856694fb-8cbc-4efe-bf5c-e3d9654da8e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.511901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856694fb-8cbc-4efe-bf5c-e3d9654da8e4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.517412 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 493/995] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Aug 11 09:41:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:54.523263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856694fb-8cbc-4efe-bf5c-e3d9654da8e4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:54.591173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:54.595370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e472b-2054-406e-9dc6-b088582d2503 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.626519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856694fb-8cbc-4efe-bf5c-e3d9654da8e4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.633437 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806078c7-9c50-438a-b6bf-616c3d4e6a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.634051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806078c7-9c50-438a-b6bf-616c3d4e6a02 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.634329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806078c7-9c50-438a-b6bf-616c3d4e6a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.634602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806078c7-9c50-438a-b6bf-616c3d4e6a02 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.635236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806078c7-9c50-438a-b6bf-616c3d4e6a02 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:54.635535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806078c7-9c50-438a-b6bf-616c3d4e6a02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.636894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806078c7-9c50-438a-b6bf-616c3d4e6a02 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.646970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 503/996] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:54.652919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806078c7-9c50-438a-b6bf-616c3d4e6a02 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:54.663990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2b3d-8883-4a40-8455-7427d20d4fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.664602 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2b3d-8883-4a40-8455-7427d20d4fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.664879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2b3d-8883-4a40-8455-7427d20d4fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.665257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2b3d-8883-4a40-8455-7427d20d4fb8 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.666035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c2b3d-8883-4a40-8455-7427d20d4fb8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:54.666317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c2b3d-8883-4a40-8455-7427d20d4fb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.667649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c2b3d-8883-4a40-8455-7427d20d4fb8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.672895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 494/997] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Aug 11 09:41:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:54.678449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2b3d-8883-4a40-8455-7427d20d4fb8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:54.692853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:54.765451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806078c7-9c50-438a-b6bf-616c3d4e6a02 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.784722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2b3d-8883-4a40-8455-7427d20d4fb8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.802786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc96c0-5d95-4e81-8544-f44524f8f656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.803636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc96c0-5d95-4e81-8544-f44524f8f656 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.804178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc96c0-5d95-4e81-8544-f44524f8f656 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.804447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc96c0-5d95-4e81-8544-f44524f8f656 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.805179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc96c0-5d95-4e81-8544-f44524f8f656 None admin] RBAC: Adding query filter params (interface=public, service_id=31eddf3ef3d44201bb6b882067e2e8e3, region_id=RegionOne) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:54.805493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc96c0-5d95-4e81-8544-f44524f8f656 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.806953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc96c0-5d95-4e81-8544-f44524f8f656 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.812356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 504/998] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Aug 11 09:41:54 2022] GET /identity/v3/endpoints?service_id=31eddf3ef3d44201bb6b882067e2e8e3&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:54.818270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc96c0-5d95-4e81-8544-f44524f8f656 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:54.820992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64881396-4482-499b-879b-03bdcc0c452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.821641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881396-4482-499b-879b-03bdcc0c452a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.821921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881396-4482-499b-879b-03bdcc0c452a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.822195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881396-4482-499b-879b-03bdcc0c452a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.822908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64881396-4482-499b-879b-03bdcc0c452a None admin] RBAC: Adding query filter params (interface=admin, service_id=61bbb8ceb36d45db9e56ef342c361de8, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:54.823215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64881396-4482-499b-879b-03bdcc0c452a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.824555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64881396-4482-499b-879b-03bdcc0c452a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.829797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 495/999] 10.222.0.17 () {56 vars in 1249 bytes} [Thu Aug 11 09:41:54 2022] GET /identity/v3/endpoints?service_id=61bbb8ceb36d45db9e56ef342c361de8&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:54.836383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64881396-4482-499b-879b-03bdcc0c452a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:54.852680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:54.916683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:54.930911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc96c0-5d95-4e81-8544-f44524f8f656 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.946386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64881396-4482-499b-879b-03bdcc0c452a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:54.973734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6eb343-35c4-45c2-bc28-e42ca0e48a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.974344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6eb343-35c4-45c2-bc28-e42ca0e48a1f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.974627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6eb343-35c4-45c2-bc28-e42ca0e48a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.974903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6eb343-35c4-45c2-bc28-e42ca0e48a1f None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.975513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6eb343-35c4-45c2-bc28-e42ca0e48a1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.976872 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6eb343-35c4-45c2-bc28-e42ca0e48a1f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.982068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859c8ce-145b-4473-a51b-557949f4771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:54.982676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859c8ce-145b-4473-a51b-557949f4771d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:54.982957 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859c8ce-145b-4473-a51b-557949f4771d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:54.993367 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859c8ce-145b-4473-a51b-557949f4771d None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:54.993610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 505/1000] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Aug 11 09:41:54 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:41:54.993716 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859c8ce-145b-4473-a51b-557949f4771d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:54.993716 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859c8ce-145b-4473-a51b-557949f4771d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:54.999510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 496/1001] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Aug 11 09:41:54 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 09:41:56.186219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bcc4f-e235-4ecc-8331-72eb6be4aa2a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:56.186219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bcc4f-e235-4ecc-8331-72eb6be4aa2a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:56.186219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3bcc4f-e235-4ecc-8331-72eb6be4aa2a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:56.187575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 506/1002] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:56.193607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad5063-1870-4323-bdc2-2107d543ac6f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:56.194066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad5063-1870-4323-bdc2-2107d543ac6f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:56.194253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad5063-1870-4323-bdc2-2107d543ac6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:56.213166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-95ad5063-1870-4323-bdc2-2107d543ac6f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:56.214357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-95ad5063-1870-4323-bdc2-2107d543ac6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:56.282910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 497/1003] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:56.306062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bcc4f-e235-4ecc-8331-72eb6be4aa2a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:56.398467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:56.409251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3bcc4f-e235-4ecc-8331-72eb6be4aa2a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:56.443537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909c5e6-a802-4200-a3a7-e1b278f5783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:56.444125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909c5e6-a802-4200-a3a7-e1b278f5783c None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:56.444410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909c5e6-a802-4200-a3a7-e1b278f5783c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:56.444685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909c5e6-a802-4200-a3a7-e1b278f5783c None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:56.445357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909c5e6-a802-4200-a3a7-e1b278f5783c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:56.446715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909c5e6-a802-4200-a3a7-e1b278f5783c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:56.568077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 507/1004] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:41:56 2022] POST /identity/v3/services => generated 242 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:56.996257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc9450-0309-4fb2-98b7-7970e7525092 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:56.996656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc9450-0309-4fb2-98b7-7970e7525092 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:56.996840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc9450-0309-4fb2-98b7-7970e7525092 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:56.998321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 498/1005] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:57.003673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da04306-a4c9-4255-87eb-1405c6dbad75 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:57.003960 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da04306-a4c9-4255-87eb-1405c6dbad75 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:57.004233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da04306-a4c9-4255-87eb-1405c6dbad75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:57.022953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4da04306-a4c9-4255-87eb-1405c6dbad75 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:57.024152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4da04306-a4c9-4255-87eb-1405c6dbad75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:57.079966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e420d0-ec4d-42cf-84ef-561a4824ff8b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:57.080348 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e420d0-ec4d-42cf-84ef-561a4824ff8b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:57.080724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e420d0-ec4d-42cf-84ef-561a4824ff8b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:57.082229 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 499/1006] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:57.087052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681a4ea-6dae-4036-ae75-2ae4a559d8a2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:57.087340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681a4ea-6dae-4036-ae75-2ae4a559d8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:57.087618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681a4ea-6dae-4036-ae75-2ae4a559d8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:57.102823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 508/1007] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:57 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:57.110452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9681a4ea-6dae-4036-ae75-2ae4a559d8a2 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:57.112165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9681a4ea-6dae-4036-ae75-2ae4a559d8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:57.126238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da04306-a4c9-4255-87eb-1405c6dbad75 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:57.179851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:57.191812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 500/1008] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:57 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:57.210723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:57.212618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681a4ea-6dae-4036-ae75-2ae4a559d8a2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:57.234474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da04306-a4c9-4255-87eb-1405c6dbad75 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:57.271977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee2b86-5242-4335-bbb6-87ce6660a483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:57.271977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2b86-5242-4335-bbb6-87ce6660a483 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:57.271977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2b86-5242-4335-bbb6-87ce6660a483 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:57.272340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee2b86-5242-4335-bbb6-87ce6660a483 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:57.272975 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee2b86-5242-4335-bbb6-87ce6660a483 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:57.274378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee2b86-5242-4335-bbb6-87ce6660a483 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-e9ee2b86-5242-4335-bbb6-87ce6660a483 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:57.283155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:57.284321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:57.286248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 509/1009] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:41:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:57.291313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee2b86-5242-4335-bbb6-87ce6660a483 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:57.321829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681a4ea-6dae-4036-ae75-2ae4a559d8a2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:57.357276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee46b0-f6ee-4795-b208-f9f8c8291825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:57.357908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee46b0-f6ee-4795-b208-f9f8c8291825 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:57.358187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee46b0-f6ee-4795-b208-f9f8c8291825 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:57.358444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee46b0-f6ee-4795-b208-f9f8c8291825 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:57.359029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee46b0-f6ee-4795-b208-f9f8c8291825 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:57.360362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee46b0-f6ee-4795-b208-f9f8c8291825 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0fee46b0-f6ee-4795-b208-f9f8c8291825 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:57.369465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:57.370460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:41:57.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 501/1010] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Aug 11 09:41:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:41:57.376482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee46b0-f6ee-4795-b208-f9f8c8291825 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:57.403460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee2b86-5242-4335-bbb6-87ce6660a483 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:57.439206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddd56c-8ec4-4d33-8e05-93f39588f478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:57.439884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddd56c-8ec4-4d33-8e05-93f39588f478 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:57.440162 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddd56c-8ec4-4d33-8e05-93f39588f478 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:57.440521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddd56c-8ec4-4d33-8e05-93f39588f478 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:57.441190 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddd56c-8ec4-4d33-8e05-93f39588f478 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:57.441493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddd56c-8ec4-4d33-8e05-93f39588f478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:57.442496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:57.443095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddd56c-8ec4-4d33-8e05-93f39588f478 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:57.452648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 510/1011] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:57.453683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddd56c-8ec4-4d33-8e05-93f39588f478 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:57.463011 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:57.486071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee46b0-f6ee-4795-b208-f9f8c8291825 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:57.521921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0814c-3bfc-47e5-a7df-04f84f1494f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:57.522724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0814c-3bfc-47e5-a7df-04f84f1494f3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:57.522954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0814c-3bfc-47e5-a7df-04f84f1494f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:57.523240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0814c-3bfc-47e5-a7df-04f84f1494f3 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:57.523871 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0814c-3bfc-47e5-a7df-04f84f1494f3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:57.524177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0814c-3bfc-47e5-a7df-04f84f1494f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:57.525554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0814c-3bfc-47e5-a7df-04f84f1494f3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:57.535539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 502/1012] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Aug 11 09:41:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:57.537389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0814c-3bfc-47e5-a7df-04f84f1494f3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:57.562051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddd56c-8ec4-4d33-8e05-93f39588f478 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:57.597044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1054a-54a3-424c-95d9-cbd184439de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:57.597704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1054a-54a3-424c-95d9-cbd184439de9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:57.597978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1054a-54a3-424c-95d9-cbd184439de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:57.598252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1054a-54a3-424c-95d9-cbd184439de9 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:57.598884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1054a-54a3-424c-95d9-cbd184439de9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:57.599182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1054a-54a3-424c-95d9-cbd184439de9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:57.600527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1054a-54a3-424c-95d9-cbd184439de9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:57.605471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 511/1013] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Aug 11 09:41:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:57.611366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1054a-54a3-424c-95d9-cbd184439de9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:57.650214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0814c-3bfc-47e5-a7df-04f84f1494f3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:57.679260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:57.689236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6c05-9c90-43d1-b111-22f5bb194f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:57.690449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6c05-9c90-43d1-b111-22f5bb194f5f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:57.690735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6c05-9c90-43d1-b111-22f5bb194f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:57.691011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe6c05-9c90-43d1-b111-22f5bb194f5f None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:57.691683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe6c05-9c90-43d1-b111-22f5bb194f5f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:57.692056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe6c05-9c90-43d1-b111-22f5bb194f5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:57.693722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe6c05-9c90-43d1-b111-22f5bb194f5f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:57.694422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:57.703507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 503/1014] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Aug 11 09:41:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:41:57.704489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6c05-9c90-43d1-b111-22f5bb194f5f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:57.729327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1054a-54a3-424c-95d9-cbd184439de9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:57.766759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380605e-eec8-45aa-8399-0c3c8c9e1554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:57.767366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380605e-eec8-45aa-8399-0c3c8c9e1554 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:57.767645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380605e-eec8-45aa-8399-0c3c8c9e1554 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:57.768636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380605e-eec8-45aa-8399-0c3c8c9e1554 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:57.769325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380605e-eec8-45aa-8399-0c3c8c9e1554 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:57.770685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380605e-eec8-45aa-8399-0c3c8c9e1554 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:57.806657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe6c05-9c90-43d1-b111-22f5bb194f5f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:57.840569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a6380-b1e1-4ec6-a024-2050b4081074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:57.841216 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a6380-b1e1-4ec6-a024-2050b4081074 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:57.841496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a6380-b1e1-4ec6-a024-2050b4081074 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:57.841767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a6380-b1e1-4ec6-a024-2050b4081074 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:57.842416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a6380-b1e1-4ec6-a024-2050b4081074 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:57.843968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a6380-b1e1-4ec6-a024-2050b4081074 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:57.852782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 512/1015] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:41:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:57.932389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 504/1016] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:41:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:41:58.605119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c851f3f-ef89-4dc2-8530-ec312b353689 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:58.605674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c851f3f-ef89-4dc2-8530-ec312b353689 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:58.605674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c851f3f-ef89-4dc2-8530-ec312b353689 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:58.607007 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 513/1017] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:58.612502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b66acf-547f-421e-bf1f-85de84fb7817 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:58.612961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b66acf-547f-421e-bf1f-85de84fb7817 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:58.613217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b66acf-547f-421e-bf1f-85de84fb7817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:58.633033 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c8b66acf-547f-421e-bf1f-85de84fb7817 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:41:58.634261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c8b66acf-547f-421e-bf1f-85de84fb7817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:41:58.682729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:58.711240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 505/1018] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:41:58.733284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c851f3f-ef89-4dc2-8530-ec312b353689 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:58.834561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c851f3f-ef89-4dc2-8530-ec312b353689 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:58.859381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:58.868190 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed106ca-0391-4405-8e35-e7e467419bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:58.868798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed106ca-0391-4405-8e35-e7e467419bcd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:58.869167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed106ca-0391-4405-8e35-e7e467419bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:58.869385 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed106ca-0391-4405-8e35-e7e467419bcd None admin] PATH_INFO: `/v3/services/image` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:58.869950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed106ca-0391-4405-8e35-e7e467419bcd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:58.871305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed106ca-0391-4405-8e35-e7e467419bcd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-aed106ca-0391-4405-8e35-e7e467419bcd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:41:58.880359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:41:58.881556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 09:41:58.882754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:41:58.885363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 514/1019] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Aug 11 09:41:58 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:41:58.887334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b66acf-547f-421e-bf1f-85de84fb7817 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:58.989532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b66acf-547f-421e-bf1f-85de84fb7817 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:59.024660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c706f-9633-4d49-82ee-048a7eee62a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:59.025309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c706f-9633-4d49-82ee-048a7eee62a3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:59.025593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c706f-9633-4d49-82ee-048a7eee62a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:59.025866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c706f-9633-4d49-82ee-048a7eee62a3 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:59.026556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c706f-9633-4d49-82ee-048a7eee62a3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:59.026862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c706f-9633-4d49-82ee-048a7eee62a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:59.028232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c706f-9633-4d49-82ee-048a7eee62a3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:59.034927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 506/1020] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:59.039630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed106ca-0391-4405-8e35-e7e467419bcd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:59.150793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed106ca-0391-4405-8e35-e7e467419bcd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:59.184051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381f0c8-d365-4eea-9450-4205b1bafaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:59.184658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381f0c8-d365-4eea-9450-4205b1bafaeb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:59.184937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381f0c8-d365-4eea-9450-4205b1bafaeb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:59.185230 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381f0c8-d365-4eea-9450-4205b1bafaeb None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:59.185853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381f0c8-d365-4eea-9450-4205b1bafaeb None admin] RBAC: Adding query filter params (type=image) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:59.186149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381f0c8-d365-4eea-9450-4205b1bafaeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:59.187537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381f0c8-d365-4eea-9450-4205b1bafaeb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:59.199995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 515/1021] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:41:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:59.201744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c706f-9633-4d49-82ee-048a7eee62a3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:59.245617 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:59.303176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c706f-9633-4d49-82ee-048a7eee62a3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:59.337022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bcbba-32e7-4074-b80a-a9b9ccbe9770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:59.337648 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bcbba-32e7-4074-b80a-a9b9ccbe9770 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:59.337930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bcbba-32e7-4074-b80a-a9b9ccbe9770 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:59.338203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bcbba-32e7-4074-b80a-a9b9ccbe9770 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:59.338899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18bcbba-32e7-4074-b80a-a9b9ccbe9770 None admin] RBAC: Adding query filter params (interface=public, service_id=8749051119ac401489bd7d3eef20036a, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:41:59.339201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18bcbba-32e7-4074-b80a-a9b9ccbe9770 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:59.340549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18bcbba-32e7-4074-b80a-a9b9ccbe9770 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:59.350078 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 507/1022] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Aug 11 09:41:59 2022] GET /identity/v3/endpoints?service_id=8749051119ac401489bd7d3eef20036a&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:41:59.351519 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381f0c8-d365-4eea-9450-4205b1bafaeb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:41:59.409802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:41:59.456178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381f0c8-d365-4eea-9450-4205b1bafaeb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:41:59.489935 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092409a-fcf6-44e7-a70e-3fb94eea1633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:41:59.490538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092409a-fcf6-44e7-a70e-3fb94eea1633 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:59.490814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092409a-fcf6-44e7-a70e-3fb94eea1633 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:59.491082 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092409a-fcf6-44e7-a70e-3fb94eea1633 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:59.491674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092409a-fcf6-44e7-a70e-3fb94eea1633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:41:59.493049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e092409a-fcf6-44e7-a70e-3fb94eea1633 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:41:59.505048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 516/1023] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Aug 11 09:41:59 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 09:41:59.983062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf1b61-e5bd-4531-9ca5-b8fd4b4fbb9d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:59.983579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf1b61-e5bd-4531-9ca5-b8fd4b4fbb9d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:59.983579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf1b61-e5bd-4531-9ca5-b8fd4b4fbb9d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:59.984987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 508/1024] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:41:59.990160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03aa17-fc67-4bdd-b3b6-64506f9a62f6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:59.990569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03aa17-fc67-4bdd-b3b6-64506f9a62f6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:59.990739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03aa17-fc67-4bdd-b3b6-64506f9a62f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:59.992691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44594fbb-b2d9-42cb-b800-b3b77e8491c9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:59.992978 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44594fbb-b2d9-42cb-b800-b3b77e8491c9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:41:59.993351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44594fbb-b2d9-42cb-b800-b3b77e8491c9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:41:59.994596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 509/1025] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:41:59.999447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee8f5-10d4-4be7-a0e8-6baeae65e272 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:41:59.999735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee8f5-10d4-4be7-a0e8-6baeae65e272 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:00.000007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee8f5-10d4-4be7-a0e8-6baeae65e272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:00.008916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7f03aa17-fc67-4bdd-b3b6-64506f9a62f6 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:00.010147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7f03aa17-fc67-4bdd-b3b6-64506f9a62f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:00.019489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-701ee8f5-10d4-4be7-a0e8-6baeae65e272 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:00.020958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-701ee8f5-10d4-4be7-a0e8-6baeae65e272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:00.083175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 517/1026] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:00.092963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 510/1027] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:41:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:42:00.110074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03aa17-fc67-4bdd-b3b6-64506f9a62f6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:00.116867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee8f5-10d4-4be7-a0e8-6baeae65e272 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:00.153961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:00.160161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:00.214846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03aa17-fc67-4bdd-b3b6-64506f9a62f6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:00.235574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee8f5-10d4-4be7-a0e8-6baeae65e272 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:00.250904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a33feb-c6b6-4269-a03b-9d41b1eff167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:00.251542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a33feb-c6b6-4269-a03b-9d41b1eff167 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:00.251817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a33feb-c6b6-4269-a03b-9d41b1eff167 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:00.252137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a33feb-c6b6-4269-a03b-9d41b1eff167 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:00.253289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a33feb-c6b6-4269-a03b-9d41b1eff167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:00.254713 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a33feb-c6b6-4269-a03b-9d41b1eff167 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:00.256442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 518/1028] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:00.261869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a33feb-c6b6-4269-a03b-9d41b1eff167 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:00.274427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a0bf3-4c9f-43c5-a7a5-f00f4b165f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:00.275056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a0bf3-4c9f-43c5-a7a5-f00f4b165f40 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:00.275340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a0bf3-4c9f-43c5-a7a5-f00f4b165f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:00.275673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a0bf3-4c9f-43c5-a7a5-f00f4b165f40 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:00.287486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a0bf3-4c9f-43c5-a7a5-f00f4b165f40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-566a0bf3-4c9f-43c5-a7a5-f00f4b165f40 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:00.292201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:00.293174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:00.294011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 511/1029] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Aug 11 09:42:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:00.299283 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a0bf3-4c9f-43c5-a7a5-f00f4b165f40 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:00.369517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a33feb-c6b6-4269-a03b-9d41b1eff167 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:00.387723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:00.397125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:00.404219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a0bf3-4c9f-43c5-a7a5-f00f4b165f40 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:00.407247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134c27c-5c92-4ed9-999a-69beec12194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:00.407247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c27c-5c92-4ed9-999a-69beec12194e None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:00.407526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c27c-5c92-4ed9-999a-69beec12194e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:00.407777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c27c-5c92-4ed9-999a-69beec12194e None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:00.408979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134c27c-5c92-4ed9-999a-69beec12194e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:00.410599 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134c27c-5c92-4ed9-999a-69beec12194e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:00.440417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22701901-5ece-4356-a7f3-6c6ea9f3ca91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:00.441061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22701901-5ece-4356-a7f3-6c6ea9f3ca91 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:00.441353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22701901-5ece-4356-a7f3-6c6ea9f3ca91 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:00.441629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22701901-5ece-4356-a7f3-6c6ea9f3ca91 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:00.442672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22701901-5ece-4356-a7f3-6c6ea9f3ca91 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:00.442975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22701901-5ece-4356-a7f3-6c6ea9f3ca91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:00.444391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22701901-5ece-4356-a7f3-6c6ea9f3ca91 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:00.452548 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 512/1030] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Aug 11 09:42:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:00.459087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22701901-5ece-4356-a7f3-6c6ea9f3ca91 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:00.503974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 519/1031] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:42:00 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:42:00.569500 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22701901-5ece-4356-a7f3-6c6ea9f3ca91 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:00.607445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6034689-4428-4aa8-abe4-45d7fbe9d86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:00.608149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6034689-4428-4aa8-abe4-45d7fbe9d86c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:00.608432 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6034689-4428-4aa8-abe4-45d7fbe9d86c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:00.608700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6034689-4428-4aa8-abe4-45d7fbe9d86c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:00.621323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6034689-4428-4aa8-abe4-45d7fbe9d86c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:00.622858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6034689-4428-4aa8-abe4-45d7fbe9d86c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d6034689-4428-4aa8-abe4-45d7fbe9d86c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:42:00.638861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:00.639891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:42:00.646008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:42:00.646008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:00.646008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:00.646008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 513/1032] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:00.648435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134c27c-5c92-4ed9-999a-69beec12194e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:00.755949 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134c27c-5c92-4ed9-999a-69beec12194e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:00.790034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8c9ff-3f71-40a4-ad88-b27200f5d69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:00.790767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8c9ff-3f71-40a4-ad88-b27200f5d69b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:00.791170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8c9ff-3f71-40a4-ad88-b27200f5d69b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:00.791556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8c9ff-3f71-40a4-ad88-b27200f5d69b None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:00.792292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8c9ff-3f71-40a4-ad88-b27200f5d69b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:00.792688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8c9ff-3f71-40a4-ad88-b27200f5d69b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:00.794163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8c9ff-3f71-40a4-ad88-b27200f5d69b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:00.806258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 520/1033] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:00.811960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6034689-4428-4aa8-abe4-45d7fbe9d86c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:00.824898 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:00.914057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6034689-4428-4aa8-abe4-45d7fbe9d86c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:00.948632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c1027-5f68-401d-b96b-8dde155b7a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:00.949293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c1027-5f68-401d-b96b-8dde155b7a6b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:00.949569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c1027-5f68-401d-b96b-8dde155b7a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:00.949837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c1027-5f68-401d-b96b-8dde155b7a6b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:00.958267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c1027-5f68-401d-b96b-8dde155b7a6b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:00.959695 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c1027-5f68-401d-b96b-8dde155b7a6b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6e3c1027-5f68-401d-b96b-8dde155b7a6b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:42:00.971152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:00.972250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:00.973974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:00.973974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:00.973974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:42:00.973974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:42:00.973974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:42:00.973974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:42:00.973974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:42:00.973974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:42:00.973974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:42:00.973974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:00.974925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 514/1034] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Aug 11 09:42:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:00.978317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8c9ff-3f71-40a4-ad88-b27200f5d69b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:01.010714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:01.086955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8c9ff-3f71-40a4-ad88-b27200f5d69b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:01.121854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19997cb0-ad2a-4be3-a00c-b536ad4eaeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:01.122464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19997cb0-ad2a-4be3-a00c-b536ad4eaeed None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:01.122747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19997cb0-ad2a-4be3-a00c-b536ad4eaeed None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:01.123022 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19997cb0-ad2a-4be3-a00c-b536ad4eaeed None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:01.123756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19997cb0-ad2a-4be3-a00c-b536ad4eaeed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:01.124061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19997cb0-ad2a-4be3-a00c-b536ad4eaeed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:01.125516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19997cb0-ad2a-4be3-a00c-b536ad4eaeed None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:01.143258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 521/1035] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Aug 11 09:42:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:01.149368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c1027-5f68-401d-b96b-8dde155b7a6b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:01.226537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:01.261097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c1027-5f68-401d-b96b-8dde155b7a6b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:01.294106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ce09de-8ce7-4d20-91ec-b54650726cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:01.294715 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce09de-8ce7-4d20-91ec-b54650726cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:01.294993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce09de-8ce7-4d20-91ec-b54650726cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:01.295264 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce09de-8ce7-4d20-91ec-b54650726cb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:01.363915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce09de-8ce7-4d20-91ec-b54650726cb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:01.365361 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ce09de-8ce7-4d20-91ec-b54650726cb3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:01.429049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:01.469167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 515/1036] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:01 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:42:01.470933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19997cb0-ad2a-4be3-a00c-b536ad4eaeed None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:01.538395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4982-ec1f-426a-8588-d2ed163079cd None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:01.538682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4982-ec1f-426a-8588-d2ed163079cd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:01.538962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4982-ec1f-426a-8588-d2ed163079cd None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:01.540441 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 516/1037] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:01.545082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9dc86-9c44-4ee1-8497-32acc3298edc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:01.545372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9dc86-9c44-4ee1-8497-32acc3298edc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:01.545656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9dc86-9c44-4ee1-8497-32acc3298edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:01.557138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:01.564903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e5c9dc86-9c44-4ee1-8497-32acc3298edc None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:01.566127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e5c9dc86-9c44-4ee1-8497-32acc3298edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:01.574248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19997cb0-ad2a-4be3-a00c-b536ad4eaeed None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:01.614852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da0acf-5ffa-4a65-bcdd-fdda16d7cb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:01.615839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da0acf-5ffa-4a65-bcdd-fdda16d7cb77 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:01.616604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da0acf-5ffa-4a65-bcdd-fdda16d7cb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:01.617209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da0acf-5ffa-4a65-bcdd-fdda16d7cb77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:01.629811 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da0acf-5ffa-4a65-bcdd-fdda16d7cb77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:01.631253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da0acf-5ffa-4a65-bcdd-fdda16d7cb77 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:01.638611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 517/1038] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-40da0acf-5ffa-4a65-bcdd-fdda16d7cb77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:42:01.647779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:01.648712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:42:01.649625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:42:01.649625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:01.649625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:01.653307 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 522/1039] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:01.654613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9dc86-9c44-4ee1-8497-32acc3298edc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:01.662044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da0acf-5ffa-4a65-bcdd-fdda16d7cb77 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:01.762114 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:01.766501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9dc86-9c44-4ee1-8497-32acc3298edc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:01.772924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da0acf-5ffa-4a65-bcdd-fdda16d7cb77 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:01.805677 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b6ff0-64a3-452b-8e9f-1d89a0739c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:01.806227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b6ff0-64a3-452b-8e9f-1d89a0739c50 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:01.806512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b6ff0-64a3-452b-8e9f-1d89a0739c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:01.806787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b6ff0-64a3-452b-8e9f-1d89a0739c50 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:01.807437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b6ff0-64a3-452b-8e9f-1d89a0739c50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:01.807743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b6ff0-64a3-452b-8e9f-1d89a0739c50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:01.809123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b6ff0-64a3-452b-8e9f-1d89a0739c50 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:01.810596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538811e1-3e25-40d4-990a-d191b3d276a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:01.811566 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538811e1-3e25-40d4-990a-d191b3d276a0 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:01.811910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538811e1-3e25-40d4-990a-d191b3d276a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:01.812187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538811e1-3e25-40d4-990a-d191b3d276a0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:01.812771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538811e1-3e25-40d4-990a-d191b3d276a0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:01.814149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538811e1-3e25-40d4-990a-d191b3d276a0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:01.825561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 518/1040] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-538811e1-3e25-40d4-990a-d191b3d276a0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:01.825741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:01.827314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:01.828405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 523/1041] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Aug 11 09:42:01 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:01.832040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b6ff0-64a3-452b-8e9f-1d89a0739c50 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:01.832560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538811e1-3e25-40d4-990a-d191b3d276a0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:01.836855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:01.938585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538811e1-3e25-40d4-990a-d191b3d276a0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:01.949734 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b6ff0-64a3-452b-8e9f-1d89a0739c50 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:01.975972 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9fa5b-76b4-4c0c-89e1-2caaaea3705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:01.976838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9fa5b-76b4-4c0c-89e1-2caaaea3705f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:01.977178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9fa5b-76b4-4c0c-89e1-2caaaea3705f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:01.977459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9fa5b-76b4-4c0c-89e1-2caaaea3705f None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:01.978091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9fa5b-76b4-4c0c-89e1-2caaaea3705f None admin] RBAC: Adding query filter params (name=image) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:01.978397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9fa5b-76b4-4c0c-89e1-2caaaea3705f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:01.979753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9fa5b-76b4-4c0c-89e1-2caaaea3705f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:01.985357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 524/1042] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:01.991125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9fa5b-76b4-4c0c-89e1-2caaaea3705f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:01.991501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc002c-f8bd-49f8-be3e-6c4afcb79534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:01.992134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc002c-f8bd-49f8-be3e-6c4afcb79534 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:01.992407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc002c-f8bd-49f8-be3e-6c4afcb79534 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:01.992680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc002c-f8bd-49f8-be3e-6c4afcb79534 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:02.000810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc002c-f8bd-49f8-be3e-6c4afcb79534 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:02.002964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc002c-f8bd-49f8-be3e-6c4afcb79534 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-bafc002c-f8bd-49f8-be3e-6c4afcb79534 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:42:02.014498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:02.015436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:02.016326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:42:02.016326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:42:02.016326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:42:02.016326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:42:02.016326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:42:02.016326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:42:02.016326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:42:02.016326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:02.016326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 519/1043] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:42:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:02.021587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc002c-f8bd-49f8-be3e-6c4afcb79534 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:02.041259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:02.064054 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:02.100787 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9fa5b-76b4-4c0c-89e1-2caaaea3705f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:02.130300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc002c-f8bd-49f8-be3e-6c4afcb79534 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:02.137974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985bd8c-cf7a-453b-82ec-af022fcd2742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:02.138593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985bd8c-cf7a-453b-82ec-af022fcd2742 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:02.138880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985bd8c-cf7a-453b-82ec-af022fcd2742 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:02.139156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985bd8c-cf7a-453b-82ec-af022fcd2742 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:02.139794 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985bd8c-cf7a-453b-82ec-af022fcd2742 None admin] RBAC: Adding query filter params (type=image) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:02.140099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985bd8c-cf7a-453b-82ec-af022fcd2742 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:02.141499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985bd8c-cf7a-453b-82ec-af022fcd2742 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:02.151420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 525/1044] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:02.152613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985bd8c-cf7a-453b-82ec-af022fcd2742 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:02.175633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb49e0-0514-4cb2-87bd-1cebcbf0d007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:02.176312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb49e0-0514-4cb2-87bd-1cebcbf0d007 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:02.177093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb49e0-0514-4cb2-87bd-1cebcbf0d007 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:02.177377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb49e0-0514-4cb2-87bd-1cebcbf0d007 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:02.178353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb49e0-0514-4cb2-87bd-1cebcbf0d007 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:02.178669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb49e0-0514-4cb2-87bd-1cebcbf0d007 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:02.183981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb49e0-0514-4cb2-87bd-1cebcbf0d007 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:02.200907 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 520/1045] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:02.201822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb49e0-0514-4cb2-87bd-1cebcbf0d007 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:02.264508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985bd8c-cf7a-453b-82ec-af022fcd2742 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:02.290704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:02.295211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:02.301584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96befbde-23f6-4708-a303-ed2f19eecfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:02.302201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96befbde-23f6-4708-a303-ed2f19eecfc7 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:02.306198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96befbde-23f6-4708-a303-ed2f19eecfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:02.306198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96befbde-23f6-4708-a303-ed2f19eecfc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:02.306198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96befbde-23f6-4708-a303-ed2f19eecfc7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:02.306198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96befbde-23f6-4708-a303-ed2f19eecfc7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:02.314576 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb49e0-0514-4cb2-87bd-1cebcbf0d007 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:02.349605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257085e-e9c7-4638-8574-762f31abbd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:02.350222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257085e-e9c7-4638-8574-762f31abbd99 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:02.350556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257085e-e9c7-4638-8574-762f31abbd99 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:02.350836 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257085e-e9c7-4638-8574-762f31abbd99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:02.351565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257085e-e9c7-4638-8574-762f31abbd99 None admin] RBAC: Adding query filter params (role.id=1a335dba62e246ec820c70f60d2558b9, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=34fa13a87ebe4b789ca1884e5aeb9237) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:02.351875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257085e-e9c7-4638-8574-762f31abbd99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:02.353574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257085e-e9c7-4638-8574-762f31abbd99 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:02.359165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 521/1046] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:42:02 2022] GET /identity/v3/role_assignments?user.id=34fa13a87ebe4b789ca1884e5aeb9237&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=1a335dba62e246ec820c70f60d2558b9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:02.381303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 526/1047] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:42:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:42:02.580330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0e2b-ea67-4a61-8465-c435cc0ed561 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:02.580545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0e2b-ea67-4a61-8465-c435cc0ed561 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:02.580927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0e2b-ea67-4a61-8465-c435cc0ed561 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:02.582450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 522/1048] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:02.587635 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067dee53-bcdb-4235-a745-2631611b2c6f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:02.588162 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067dee53-bcdb-4235-a745-2631611b2c6f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:02.588628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067dee53-bcdb-4235-a745-2631611b2c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:02.607120 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-067dee53-bcdb-4235-a745-2631611b2c6f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:02.608312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-067dee53-bcdb-4235-a745-2631611b2c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:02.685237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 527/1049] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:02.709443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee0e2b-ea67-4a61-8465-c435cc0ed561 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:02.817711 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee0e2b-ea67-4a61-8465-c435cc0ed561 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:02.851701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13834c02-2db2-4337-b8df-feac7f73e980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:02.852258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13834c02-2db2-4337-b8df-feac7f73e980 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:02.852537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13834c02-2db2-4337-b8df-feac7f73e980 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:02.852807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13834c02-2db2-4337-b8df-feac7f73e980 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:02.853961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13834c02-2db2-4337-b8df-feac7f73e980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:02.855385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13834c02-2db2-4337-b8df-feac7f73e980 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:02.862402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 523/1050] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:02.863749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067dee53-bcdb-4235-a745-2631611b2c6f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:02.964982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067dee53-bcdb-4235-a745-2631611b2c6f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:02.978772 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:03.001175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250787e-dc4a-4311-a9ae-fcdb23e0a208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:03.002103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250787e-dc4a-4311-a9ae-fcdb23e0a208 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:03.002567 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250787e-dc4a-4311-a9ae-fcdb23e0a208 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:03.003011 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250787e-dc4a-4311-a9ae-fcdb23e0a208 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:03.003869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250787e-dc4a-4311-a9ae-fcdb23e0a208 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:03.006288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250787e-dc4a-4311-a9ae-fcdb23e0a208 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:03.093048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 528/1051] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:42:02 2022] POST /identity/v3/users => generated 285 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:42:04.367229 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336c75f-5684-471a-96d0-264de6c977e6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:04.367740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336c75f-5684-471a-96d0-264de6c977e6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:04.367818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336c75f-5684-471a-96d0-264de6c977e6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:04.369166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 524/1052] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:04.374537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b473f1b-49e7-4766-a1ea-61856fb4c926 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:04.374927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b473f1b-49e7-4766-a1ea-61856fb4c926 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:04.374927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b473f1b-49e7-4766-a1ea-61856fb4c926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:04.393710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0b473f1b-49e7-4766-a1ea-61856fb4c926 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:04.394955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0b473f1b-49e7-4766-a1ea-61856fb4c926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:04.431317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba362592-fec1-41bb-a111-6abbfdda8fea None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:04.431599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba362592-fec1-41bb-a111-6abbfdda8fea None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:04.431924 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba362592-fec1-41bb-a111-6abbfdda8fea None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:04.433291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 525/1053] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:42:04.438152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ddcaf-6020-4d36-8aed-59f30845b340 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:04.438370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ddcaf-6020-4d36-8aed-59f30845b340 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:04.438649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ddcaf-6020-4d36-8aed-59f30845b340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:04.459703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d24ddcaf-6020-4d36-8aed-59f30845b340 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:04.461057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d24ddcaf-6020-4d36-8aed-59f30845b340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:04.465426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 529/1054] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:04.468434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:04.487195 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b473f1b-49e7-4766-a1ea-61856fb4c926 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:04.536189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 526/1055] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:42:04.558421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ddcaf-6020-4d36-8aed-59f30845b340 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:04.602530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:04.606172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b473f1b-49e7-4766-a1ea-61856fb4c926 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:04.643872 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc8988-3b28-4a00-81c4-b881f5ec6630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:04.644658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc8988-3b28-4a00-81c4-b881f5ec6630 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:04.644944 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc8988-3b28-4a00-81c4-b881f5ec6630 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:04.645243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc8988-3b28-4a00-81c4-b881f5ec6630 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:04.656636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc8988-3b28-4a00-81c4-b881f5ec6630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-1fcc8988-3b28-4a00-81c4-b881f5ec6630 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:04.662106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:04.663255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:42:04.664289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:04.669481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ddcaf-6020-4d36-8aed-59f30845b340 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:04.671381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 530/1056] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Aug 11 09:42:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:04.672076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc8988-3b28-4a00-81c4-b881f5ec6630 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:04.708907 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b57db-36e0-4b8c-89e4-1838af45823a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:04.709930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b57db-36e0-4b8c-89e4-1838af45823a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:04.710266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b57db-36e0-4b8c-89e4-1838af45823a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:04.710549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b57db-36e0-4b8c-89e4-1838af45823a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:04.723490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:04.775787 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc8988-3b28-4a00-81c4-b881f5ec6630 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:04.786227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b57db-36e0-4b8c-89e4-1838af45823a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:04.787897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b57db-36e0-4b8c-89e4-1838af45823a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:04.810718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d619d-4b68-41bb-8c1f-4b4381cac0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:04.811316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d619d-4b68-41bb-8c1f-4b4381cac0e4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:04.811591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d619d-4b68-41bb-8c1f-4b4381cac0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:04.811857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d619d-4b68-41bb-8c1f-4b4381cac0e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:04.812518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d619d-4b68-41bb-8c1f-4b4381cac0e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:04.812815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d619d-4b68-41bb-8c1f-4b4381cac0e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:04.814188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d619d-4b68-41bb-8c1f-4b4381cac0e4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:04.823164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 531/1057] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Aug 11 09:42:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:04.828743 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d619d-4b68-41bb-8c1f-4b4381cac0e4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:04.851118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:04.909191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 527/1058] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 11 09:42:04.910669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b57db-36e0-4b8c-89e4-1838af45823a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:04.936596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d619d-4b68-41bb-8c1f-4b4381cac0e4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:04.961030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:04.971347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b14a3d-bf48-43c0-939e-470956697f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:04.971973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b14a3d-bf48-43c0-939e-470956697f77 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:04.972251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b14a3d-bf48-43c0-939e-470956697f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:04.972523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b14a3d-bf48-43c0-939e-470956697f77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:04.983162 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b14a3d-bf48-43c0-939e-470956697f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:04.984590 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b14a3d-bf48-43c0-939e-470956697f77 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-17b14a3d-bf48-43c0-939e-470956697f77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:42:04.999339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:05.000512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:42:05.001615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:42:05.001615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:05.001615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:05.001615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 532/1059] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:05.006822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b14a3d-bf48-43c0-939e-470956697f77 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:05.035283 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b57db-36e0-4b8c-89e4-1838af45823a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:05.073467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45362400-aa14-4035-8182-156fc3d9cef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:05.074507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45362400-aa14-4035-8182-156fc3d9cef3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.075142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45362400-aa14-4035-8182-156fc3d9cef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.075414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45362400-aa14-4035-8182-156fc3d9cef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.078833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:05.088435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45362400-aa14-4035-8182-156fc3d9cef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:05.090146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45362400-aa14-4035-8182-156fc3d9cef3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-45362400-aa14-4035-8182-156fc3d9cef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:42:05.105744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:05.113621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:42:05.114688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:42:05.114688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:05.114688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:05.114688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 528/1060] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:05.114912 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b14a3d-bf48-43c0-939e-470956697f77 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:05.114994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45362400-aa14-4035-8182-156fc3d9cef3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:05.153630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb72246-dbed-4676-90e3-ba551ce6e27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:05.154231 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb72246-dbed-4676-90e3-ba551ce6e27a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.154508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb72246-dbed-4676-90e3-ba551ce6e27a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.154776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb72246-dbed-4676-90e3-ba551ce6e27a None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.155410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb72246-dbed-4676-90e3-ba551ce6e27a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:05.155706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb72246-dbed-4676-90e3-ba551ce6e27a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:05.157065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb72246-dbed-4676-90e3-ba551ce6e27a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:05.169203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 533/1061] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:05.170366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8888df-0c1e-4f6f-ace4-320392cea758 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.170643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8888df-0c1e-4f6f-ace4-320392cea758 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.170911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8888df-0c1e-4f6f-ace4-320392cea758 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.172071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 534/1062] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:05.177127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8888df-0c1e-4f6f-ace4-320392cea758 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:05.218638 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:05.228022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45362400-aa14-4035-8182-156fc3d9cef3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:05.263946 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98387f3e-ea66-4c68-b7eb-72c0223e2c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:05.264553 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98387f3e-ea66-4c68-b7eb-72c0223e2c3b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.264833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98387f3e-ea66-4c68-b7eb-72c0223e2c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.265149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98387f3e-ea66-4c68-b7eb-72c0223e2c3b None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.265797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98387f3e-ea66-4c68-b7eb-72c0223e2c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:05.266096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98387f3e-ea66-4c68-b7eb-72c0223e2c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:05.267471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98387f3e-ea66-4c68-b7eb-72c0223e2c3b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:05.277773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8888df-0c1e-4f6f-ace4-320392cea758 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:05.278813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 529/1063] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:05.288033 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57247232-13f3-4e82-ba3d-420ea3420d58 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.288639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57247232-13f3-4e82-ba3d-420ea3420d58 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.288924 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57247232-13f3-4e82-ba3d-420ea3420d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.309753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:05.310813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-57247232-13f3-4e82-ba3d-420ea3420d58 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:05.312113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-57247232-13f3-4e82-ba3d-420ea3420d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:05.315562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed151534-2282-44ad-bc2d-b084bb13d979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:05.316220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151534-2282-44ad-bc2d-b084bb13d979 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.316497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151534-2282-44ad-bc2d-b084bb13d979 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.316766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed151534-2282-44ad-bc2d-b084bb13d979 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.325153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed151534-2282-44ad-bc2d-b084bb13d979 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:05.327854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed151534-2282-44ad-bc2d-b084bb13d979 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-ed151534-2282-44ad-bc2d-b084bb13d979 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:42:05.340891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:05.342114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:05.343062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:05.343062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:05.343062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:42:05.343062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:42:05.343062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:42:05.343062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:42:05.343062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:42:05.343062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:42:05.343062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:42:05.343062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:05.343062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 535/1064] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Aug 11 09:42:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:05.348346 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed151534-2282-44ad-bc2d-b084bb13d979 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:05.399776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 530/1065] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:05.422385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57247232-13f3-4e82-ba3d-420ea3420d58 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:05.462071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed151534-2282-44ad-bc2d-b084bb13d979 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:05.508526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5db58-e52b-4870-b657-9341734ed296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:05.509098 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5db58-e52b-4870-b657-9341734ed296 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.509388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5db58-e52b-4870-b657-9341734ed296 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.509664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5db58-e52b-4870-b657-9341734ed296 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.510403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5db58-e52b-4870-b657-9341734ed296 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:05.510709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5db58-e52b-4870-b657-9341734ed296 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:05.512128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5db58-e52b-4870-b657-9341734ed296 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:05.517789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:05.531241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 536/1066] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Aug 11 09:42:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:05.536609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5db58-e52b-4870-b657-9341734ed296 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:05.545227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57247232-13f3-4e82-ba3d-420ea3420d58 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:05.587071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e7946-fa93-4743-af38-de851fec78fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:05.587714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e7946-fa93-4743-af38-de851fec78fb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.588111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e7946-fa93-4743-af38-de851fec78fb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.588385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e7946-fa93-4743-af38-de851fec78fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.592097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:05.600088 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e7946-fa93-4743-af38-de851fec78fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-612e7946-fa93-4743-af38-de851fec78fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:05.605276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:05.606195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:05.607016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 531/1067] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:42:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:05.613112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e7946-fa93-4743-af38-de851fec78fb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:05.641317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5db58-e52b-4870-b657-9341734ed296 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:05.676433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937e931-0047-4713-85ee-db6c313ab179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:05.677078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937e931-0047-4713-85ee-db6c313ab179 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.677356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937e931-0047-4713-85ee-db6c313ab179 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.677625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937e931-0047-4713-85ee-db6c313ab179 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.687235 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937e931-0047-4713-85ee-db6c313ab179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:05.688658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937e931-0047-4713-85ee-db6c313ab179 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9937e931-0047-4713-85ee-db6c313ab179 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:42:05.703700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:05.704708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:42:05.706318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:42:05.706318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:05.706318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:05.706318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 537/1068] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:05.710948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937e931-0047-4713-85ee-db6c313ab179 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:05.717785 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e7946-fa93-4743-af38-de851fec78fb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:05.752458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f1d44-ea02-4302-b2ab-fefde609a3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:05.753088 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f1d44-ea02-4302-b2ab-fefde609a3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.753372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f1d44-ea02-4302-b2ab-fefde609a3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.753642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f1d44-ea02-4302-b2ab-fefde609a3b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.754297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f1d44-ea02-4302-b2ab-fefde609a3b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:05.754596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f1d44-ea02-4302-b2ab-fefde609a3b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:05.755956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f1d44-ea02-4302-b2ab-fefde609a3b7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:05.763782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 532/1069] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:42:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:05.770106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f1d44-ea02-4302-b2ab-fefde609a3b7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:05.773516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:05.808491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937e931-0047-4713-85ee-db6c313ab179 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:05.810070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:05.843479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c79478-301d-440c-8d1b-1fa2ee0c0827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:05.844077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c79478-301d-440c-8d1b-1fa2ee0c0827 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.844357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c79478-301d-440c-8d1b-1fa2ee0c0827 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.844634 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c79478-301d-440c-8d1b-1fa2ee0c0827 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.845309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c79478-301d-440c-8d1b-1fa2ee0c0827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:05.845685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c79478-301d-440c-8d1b-1fa2ee0c0827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:05.847138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c79478-301d-440c-8d1b-1fa2ee0c0827 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:05.858541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 538/1070] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:05.865045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c79478-301d-440c-8d1b-1fa2ee0c0827 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:05.869008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f1d44-ea02-4302-b2ab-fefde609a3b7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:05.903225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd4bd9-216c-4385-9349-cba737dc2c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:05.903887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd4bd9-216c-4385-9349-cba737dc2c89 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:05.904165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd4bd9-216c-4385-9349-cba737dc2c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:05.904438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd4bd9-216c-4385-9349-cba737dc2c89 None admin] PATH_INFO: `/v3/users/864f9e99afdc41b0a21c7914460b2a4d` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:05.922096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd4bd9-216c-4385-9349-cba737dc2c89 None admin] RBAC: Authorizing `identity:get_user(user_id=864f9e99afdc41b0a21c7914460b2a4d)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:05.923512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd4bd9-216c-4385-9349-cba737dc2c89 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:05.925851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 533/1071] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:05 2022] GET /identity/v3/users/864f9e99afdc41b0a21c7914460b2a4d => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:05.931403 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd4bd9-216c-4385-9349-cba737dc2c89 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:05.965115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c79478-301d-440c-8d1b-1fa2ee0c0827 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:05.991116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:05.999726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d096469-65b7-4e21-af38-94fa76b19c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:06.000354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d096469-65b7-4e21-af38-94fa76b19c70 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:06.000628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d096469-65b7-4e21-af38-94fa76b19c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:06.003157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d096469-65b7-4e21-af38-94fa76b19c70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:06.007444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d096469-65b7-4e21-af38-94fa76b19c70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:06.008869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d096469-65b7-4e21-af38-94fa76b19c70 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-9d096469-65b7-4e21-af38-94fa76b19c70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:42:06.020762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:06.021813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:06.022697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:42:06.022697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:42:06.022697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:42:06.022697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:42:06.022697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:42:06.022697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:42:06.022697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:42:06.022697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:06.022697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 539/1072] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:42:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:06.027426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d096469-65b7-4e21-af38-94fa76b19c70 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:06.030133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd4bd9-216c-4385-9349-cba737dc2c89 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:06.051536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:06.065279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d361c-257e-41a3-9139-5c70b7ae2ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:06.065883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d361c-257e-41a3-9139-5c70b7ae2ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:06.066164 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d361c-257e-41a3-9139-5c70b7ae2ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:06.066438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d361c-257e-41a3-9139-5c70b7ae2ef4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:06.125970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d096469-65b7-4e21-af38-94fa76b19c70 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:06.133573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d361c-257e-41a3-9139-5c70b7ae2ef4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:06.135007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d361c-257e-41a3-9139-5c70b7ae2ef4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:06.161395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9f2d4-2245-4fe2-8666-a70e12db4531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:06.161995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9f2d4-2245-4fe2-8666-a70e12db4531 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:06.162274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9f2d4-2245-4fe2-8666-a70e12db4531 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:06.162545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9f2d4-2245-4fe2-8666-a70e12db4531 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:06.163239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9f2d4-2245-4fe2-8666-a70e12db4531 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:06.163540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9f2d4-2245-4fe2-8666-a70e12db4531 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:06.164950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9f2d4-2245-4fe2-8666-a70e12db4531 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:06.176126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 540/1073] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:42:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:06.181934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9f2d4-2245-4fe2-8666-a70e12db4531 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:06.195881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:06.234909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 534/1074] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 11 09:42:06.240672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d361c-257e-41a3-9139-5c70b7ae2ef4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:06.264244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:06.282672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9f2d4-2245-4fe2-8666-a70e12db4531 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:06.317871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae349f45-a968-4a19-b6b3-a211a29370a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:06.318519 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae349f45-a968-4a19-b6b3-a211a29370a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:06.318795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae349f45-a968-4a19-b6b3-a211a29370a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:06.319065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae349f45-a968-4a19-b6b3-a211a29370a0 None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/34fa13a87ebe4b789ca1884e5aeb9237/roles/1a335dba62e246ec820c70f60d2558b9` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:06.321224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae349f45-a968-4a19-b6b3-a211a29370a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=34fa13a87ebe4b789ca1884e5aeb9237, role_id=1a335dba62e246ec820c70f60d2558b9)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:06.322650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae349f45-a968-4a19-b6b3-a211a29370a0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:06.345250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d361c-257e-41a3-9139-5c70b7ae2ef4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:06.375836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 541/1075] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:42:06 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/34fa13a87ebe4b789ca1884e5aeb9237/roles/1a335dba62e246ec820c70f60d2558b9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 09:42:06.379136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5e316-e659-49b6-badf-cce69283ab16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:06.379781 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5e316-e659-49b6-badf-cce69283ab16 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:06.380061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5e316-e659-49b6-badf-cce69283ab16 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:06.380333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5e316-e659-49b6-badf-cce69283ab16 None admin] PATH_INFO: `/v3/projects/fb81afa93d6f4924b0d150937d380722` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:06.381521 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5e316-e659-49b6-badf-cce69283ab16 None admin] RBAC: Authorizing `identity:get_project(project_id=fb81afa93d6f4924b0d150937d380722)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:06.382966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5e316-e659-49b6-badf-cce69283ab16 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:06.384675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 535/1076] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:06 2022] GET /identity/v3/projects/fb81afa93d6f4924b0d150937d380722 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:06.391572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae349f45-a968-4a19-b6b3-a211a29370a0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:06.498760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae349f45-a968-4a19-b6b3-a211a29370a0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:06.533544 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833b493-dd8c-4188-a30b-51d1400b196e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:06.534083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833b493-dd8c-4188-a30b-51d1400b196e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:06.534362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833b493-dd8c-4188-a30b-51d1400b196e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:06.534634 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833b493-dd8c-4188-a30b-51d1400b196e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:06.535358 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833b493-dd8c-4188-a30b-51d1400b196e None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=fb81afa93d6f4924b0d150937d380722, user.id=864f9e99afdc41b0a21c7914460b2a4d) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:06.535661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833b493-dd8c-4188-a30b-51d1400b196e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:06.537102 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833b493-dd8c-4188-a30b-51d1400b196e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:06.542463 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 542/1077] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:42:06 2022] GET /identity/v3/role_assignments?user.id=864f9e99afdc41b0a21c7914460b2a4d&scope.project.id=fb81afa93d6f4924b0d150937d380722&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:08.369078 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361dafd-689c-4551-8737-fdb7721c289d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:08.369593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361dafd-689c-4551-8737-fdb7721c289d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:08.369698 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361dafd-689c-4551-8737-fdb7721c289d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:08.370961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 536/1078] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:08.376283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf084-8fdb-4c0d-8678-14d3e6ed8646 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:08.376709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf084-8fdb-4c0d-8678-14d3e6ed8646 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:08.376850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faaf084-8fdb-4c0d-8678-14d3e6ed8646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:08.389684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:08.395422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-8faaf084-8fdb-4c0d-8678-14d3e6ed8646 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:08.396617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8faaf084-8fdb-4c0d-8678-14d3e6ed8646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:08.462715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 543/1079] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:08.486306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361dafd-689c-4551-8737-fdb7721c289d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:08.509325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1981e9a-8f3a-4d39-97ca-cdaf4c5d90d3 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:08.509615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1981e9a-8f3a-4d39-97ca-cdaf4c5d90d3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:08.509892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1981e9a-8f3a-4d39-97ca-cdaf4c5d90d3 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:08.511216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 544/1080] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:42:08.516509 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef66146-f9b1-45ee-8533-a4936176cabc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:08.516799 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef66146-f9b1-45ee-8533-a4936176cabc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:08.517026 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:08.517142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef66146-f9b1-45ee-8533-a4936176cabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:08.535027 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0ef66146-f9b1-45ee-8533-a4936176cabc None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:08.536217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0ef66146-f9b1-45ee-8533-a4936176cabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:08.590059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361dafd-689c-4551-8737-fdb7721c289d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:08.603619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 545/1081] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:42:08.624214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc7c3c-f19d-4986-94b5-c464475884f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:08.624988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc7c3c-f19d-4986-94b5-c464475884f7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:08.625305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc7c3c-f19d-4986-94b5-c464475884f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:08.625579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc7c3c-f19d-4986-94b5-c464475884f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:08.625675 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef66146-f9b1-45ee-8533-a4936176cabc None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:08.636854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc7c3c-f19d-4986-94b5-c464475884f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-eacc7c3c-f19d-4986-94b5-c464475884f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:08.641597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:08.642544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:08.643375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 537/1082] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Aug 11 09:42:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:08.648619 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc7c3c-f19d-4986-94b5-c464475884f7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:08.693100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:08.726620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef66146-f9b1-45ee-8533-a4936176cabc None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:08.740167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:08.749507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc7c3c-f19d-4986-94b5-c464475884f7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:08.760838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce3ba9-5c3b-4e8d-842c-db480411083b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:08.761501 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3ba9-5c3b-4e8d-842c-db480411083b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:08.761783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3ba9-5c3b-4e8d-842c-db480411083b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:08.762058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce3ba9-5c3b-4e8d-842c-db480411083b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:08.772759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce3ba9-5c3b-4e8d-842c-db480411083b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-30ce3ba9-5c3b-4e8d-842c-db480411083b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:08.777495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:08.778611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:08.779461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 546/1083] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:42:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:08.784795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce3ba9-5c3b-4e8d-842c-db480411083b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:08.785905 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b81fd6-0836-4fd5-bc16-ef6352df5a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:08.786513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81fd6-0836-4fd5-bc16-ef6352df5a88 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:08.786788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81fd6-0836-4fd5-bc16-ef6352df5a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:08.787061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b81fd6-0836-4fd5-bc16-ef6352df5a88 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:08.787719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b81fd6-0836-4fd5-bc16-ef6352df5a88 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:08.788020 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b81fd6-0836-4fd5-bc16-ef6352df5a88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:08.789389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b81fd6-0836-4fd5-bc16-ef6352df5a88 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:08.797289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 538/1084] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Aug 11 09:42:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:08.803406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b81fd6-0836-4fd5-bc16-ef6352df5a88 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:08.883196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce3ba9-5c3b-4e8d-842c-db480411083b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:08.902333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b81fd6-0836-4fd5-bc16-ef6352df5a88 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:08.907201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:08.916423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df2451-d23c-4db5-9156-ca60de927ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:08.917045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df2451-d23c-4db5-9156-ca60de927ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:08.917334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df2451-d23c-4db5-9156-ca60de927ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:08.917608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df2451-d23c-4db5-9156-ca60de927ce0 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:08.918314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df2451-d23c-4db5-9156-ca60de927ce0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:08.918761 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df2451-d23c-4db5-9156-ca60de927ce0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:08.920137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df2451-d23c-4db5-9156-ca60de927ce0 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:08.928721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 547/1085] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:42:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:08.934598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df2451-d23c-4db5-9156-ca60de927ce0 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:08.937383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eaabd4-d66b-488a-a0cb-4d8e246baadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:08.937892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eaabd4-d66b-488a-a0cb-4d8e246baadd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:08.938771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eaabd4-d66b-488a-a0cb-4d8e246baadd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:08.938771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eaabd4-d66b-488a-a0cb-4d8e246baadd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:08.949409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eaabd4-d66b-488a-a0cb-4d8e246baadd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:08.950836 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eaabd4-d66b-488a-a0cb-4d8e246baadd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-81eaabd4-d66b-488a-a0cb-4d8e246baadd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:42:08.964983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:08.965934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:42:08.966899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:42:08.966899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:08.966899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:08.966899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 539/1086] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:08.972519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eaabd4-d66b-488a-a0cb-4d8e246baadd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:08.975959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:09.033562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df2451-d23c-4db5-9156-ca60de927ce0 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:09.066653 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671549-6b12-4cc0-9e05-19f58aeecc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:09.067529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671549-6b12-4cc0-9e05-19f58aeecc81 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:09.067809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671549-6b12-4cc0-9e05-19f58aeecc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:09.068083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e671549-6b12-4cc0-9e05-19f58aeecc81 None admin] PATH_INFO: `/v3/users/864f9e99afdc41b0a21c7914460b2a4d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:09.069243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e671549-6b12-4cc0-9e05-19f58aeecc81 None admin] RBAC: Authorizing `identity:get_user(user_id=864f9e99afdc41b0a21c7914460b2a4d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:09.070658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e671549-6b12-4cc0-9e05-19f58aeecc81 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:09.072219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 548/1087] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:08 2022] GET /identity/v3/users/864f9e99afdc41b0a21c7914460b2a4d => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:09.075474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eaabd4-d66b-488a-a0cb-4d8e246baadd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:09.077951 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671549-6b12-4cc0-9e05-19f58aeecc81 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:09.109633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42957ffc-47e4-46d7-9a72-83d8d983b9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:09.110234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42957ffc-47e4-46d7-9a72-83d8d983b9df None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:09.110515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42957ffc-47e4-46d7-9a72-83d8d983b9df None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:09.110789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42957ffc-47e4-46d7-9a72-83d8d983b9df None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:09.111546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42957ffc-47e4-46d7-9a72-83d8d983b9df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:09.111853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42957ffc-47e4-46d7-9a72-83d8d983b9df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:09.115066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42957ffc-47e4-46d7-9a72-83d8d983b9df None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:09.127403 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 540/1088] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:09.132939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:09.133221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42957ffc-47e4-46d7-9a72-83d8d983b9df None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:09.177525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e671549-6b12-4cc0-9e05-19f58aeecc81 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:09.197404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:09.213453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267575b-879b-4ea3-a46a-26b2d3f5a0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:09.214080 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267575b-879b-4ea3-a46a-26b2d3f5a0de None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:09.214358 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267575b-879b-4ea3-a46a-26b2d3f5a0de None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:09.214644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267575b-879b-4ea3-a46a-26b2d3f5a0de None admin] PATH_INFO: `/v3/projects/fb81afa93d6f4924b0d150937d380722` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:09.215805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267575b-879b-4ea3-a46a-26b2d3f5a0de None admin] RBAC: Authorizing `identity:get_project(project_id=fb81afa93d6f4924b0d150937d380722)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:09.217254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267575b-879b-4ea3-a46a-26b2d3f5a0de None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:09.218867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 549/1089] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:09 2022] GET /identity/v3/projects/fb81afa93d6f4924b0d150937d380722 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:09.224933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267575b-879b-4ea3-a46a-26b2d3f5a0de None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:09.237709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42957ffc-47e4-46d7-9a72-83d8d983b9df None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:09.272959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758befa8-b3fe-435b-8cbe-7dad941acfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:09.273623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758befa8-b3fe-435b-8cbe-7dad941acfee None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:09.273907 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758befa8-b3fe-435b-8cbe-7dad941acfee None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:09.274179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758befa8-b3fe-435b-8cbe-7dad941acfee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:09.281683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758befa8-b3fe-435b-8cbe-7dad941acfee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:09.283143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758befa8-b3fe-435b-8cbe-7dad941acfee None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-758befa8-b3fe-435b-8cbe-7dad941acfee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:42:09.294998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:09.296100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:09.297255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:09.297255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:09.297255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:42:09.297255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:42:09.297255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:42:09.297255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:42:09.297255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:42:09.297255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:42:09.297255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 09:42:09.297255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:09.297255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 541/1090] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Aug 11 09:42:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:09.302773 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758befa8-b3fe-435b-8cbe-7dad941acfee None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:09.326494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267575b-879b-4ea3-a46a-26b2d3f5a0de None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:09.349246 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:09.360026 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad46609-0b88-4d71-b2b5-baccb2ca41ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:09.360679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad46609-0b88-4d71-b2b5-baccb2ca41ac None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:09.361524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad46609-0b88-4d71-b2b5-baccb2ca41ac None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:09.361804 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad46609-0b88-4d71-b2b5-baccb2ca41ac None admin] PATH_INFO: `/v3/projects/fb81afa93d6f4924b0d150937d380722/users/864f9e99afdc41b0a21c7914460b2a4d/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:09.363977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad46609-0b88-4d71-b2b5-baccb2ca41ac None admin] RBAC: Authorizing `identity:create_grant(project_id=fb81afa93d6f4924b0d150937d380722, user_id=864f9e99afdc41b0a21c7914460b2a4d, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:09.365436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad46609-0b88-4d71-b2b5-baccb2ca41ac None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:09.401656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758befa8-b3fe-435b-8cbe-7dad941acfee None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:09.424606 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:09.435066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d6c30-62d9-49dd-a04c-477eaeb3b927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:09.435673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d6c30-62d9-49dd-a04c-477eaeb3b927 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:09.435954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d6c30-62d9-49dd-a04c-477eaeb3b927 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:09.436228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d6c30-62d9-49dd-a04c-477eaeb3b927 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:09.436947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d6c30-62d9-49dd-a04c-477eaeb3b927 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:09.437308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d6c30-62d9-49dd-a04c-477eaeb3b927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:09.438692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d6c30-62d9-49dd-a04c-477eaeb3b927 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:09.438791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 550/1091] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:42:09 2022] PUT /identity/v3/projects/fb81afa93d6f4924b0d150937d380722/users/864f9e99afdc41b0a21c7914460b2a4d/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 09:42:09.458774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 542/1092] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Aug 11 09:42:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:09.464850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad46609-0b88-4d71-b2b5-baccb2ca41ac None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:09.569646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad46609-0b88-4d71-b2b5-baccb2ca41ac None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:09.607931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b0ca3-daee-4e48-ace5-6a240b50ad2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:09.608540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b0ca3-daee-4e48-ace5-6a240b50ad2f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:09.608816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b0ca3-daee-4e48-ace5-6a240b50ad2f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:09.609152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b0ca3-daee-4e48-ace5-6a240b50ad2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:09.660704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:09.675972 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b0ca3-daee-4e48-ace5-6a240b50ad2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:09.677494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b0ca3-daee-4e48-ace5-6a240b50ad2f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:09.774586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 551/1093] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:42:09.780308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d6c30-62d9-49dd-a04c-477eaeb3b927 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:09.875623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d6c30-62d9-49dd-a04c-477eaeb3b927 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:09.910140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d97e12-1940-408c-af80-2f84414086a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:09.910764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d97e12-1940-408c-af80-2f84414086a9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:09.911045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d97e12-1940-408c-af80-2f84414086a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:09.911308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d97e12-1940-408c-af80-2f84414086a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:09.920942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d97e12-1940-408c-af80-2f84414086a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:09.922395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d97e12-1940-408c-af80-2f84414086a9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-17d97e12-1940-408c-af80-2f84414086a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:42:09.936271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:09.937429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:42:09.938320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:42:09.938320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:42:09.938320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:09.938320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 543/1094] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:42:09.943841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b0ca3-daee-4e48-ace5-6a240b50ad2f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:10.024765 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:10.039592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b0ca3-daee-4e48-ace5-6a240b50ad2f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:10.073727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703bb87a-17d6-44e5-830d-0974f0696f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:10.074325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bb87a-17d6-44e5-830d-0974f0696f6f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:10.074600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bb87a-17d6-44e5-830d-0974f0696f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:10.074868 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703bb87a-17d6-44e5-830d-0974f0696f6f None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:10.075501 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703bb87a-17d6-44e5-830d-0974f0696f6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:10.075796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703bb87a-17d6-44e5-830d-0974f0696f6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:10.077175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703bb87a-17d6-44e5-830d-0974f0696f6f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:10.088418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 552/1095] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:10.093990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d97e12-1940-408c-af80-2f84414086a9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:10.128476 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:10.194807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d97e12-1940-408c-af80-2f84414086a9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:10.230597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26811c52-4691-48c4-889f-b116ea7d5969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:10.231220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811c52-4691-48c4-889f-b116ea7d5969 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:10.231494 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811c52-4691-48c4-889f-b116ea7d5969 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:10.231761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26811c52-4691-48c4-889f-b116ea7d5969 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:10.238384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26811c52-4691-48c4-889f-b116ea7d5969 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:10.239783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26811c52-4691-48c4-889f-b116ea7d5969 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-26811c52-4691-48c4-889f-b116ea7d5969 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:42:10.250728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:10.251754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:10.252712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:42:10.252712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:42:10.252712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:42:10.252712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:42:10.252712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:42:10.252712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:42:10.252712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:42:10.252712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:10.252712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 544/1096] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:42:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:10.257849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703bb87a-17d6-44e5-830d-0974f0696f6f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:10.355472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703bb87a-17d6-44e5-830d-0974f0696f6f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:10.388822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1dc9b-cd30-4434-a0c5-35c06c0d7e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:10.389385 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1dc9b-cd30-4434-a0c5-35c06c0d7e08 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:10.389667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1dc9b-cd30-4434-a0c5-35c06c0d7e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:10.389942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1dc9b-cd30-4434-a0c5-35c06c0d7e08 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:10.390636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1dc9b-cd30-4434-a0c5-35c06c0d7e08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:10.390943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1dc9b-cd30-4434-a0c5-35c06c0d7e08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:10.393083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1dc9b-cd30-4434-a0c5-35c06c0d7e08 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:10.405152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 553/1097] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:10.411747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26811c52-4691-48c4-889f-b116ea7d5969 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:10.502462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:10.509071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26811c52-4691-48c4-889f-b116ea7d5969 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:10.543751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50191a2e-33ab-46e6-84a1-97f524fdb57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:10.544421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50191a2e-33ab-46e6-84a1-97f524fdb57f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:10.544699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50191a2e-33ab-46e6-84a1-97f524fdb57f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:10.544966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50191a2e-33ab-46e6-84a1-97f524fdb57f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:10.545728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50191a2e-33ab-46e6-84a1-97f524fdb57f None admin] RBAC: Adding query filter params (role.id=1a335dba62e246ec820c70f60d2558b9, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=34fa13a87ebe4b789ca1884e5aeb9237) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:10.546032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50191a2e-33ab-46e6-84a1-97f524fdb57f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:10.547424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50191a2e-33ab-46e6-84a1-97f524fdb57f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:10.553491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 545/1098] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:42:10 2022] GET /identity/v3/role_assignments?user.id=34fa13a87ebe4b789ca1884e5aeb9237&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=1a335dba62e246ec820c70f60d2558b9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:42:11.508906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df89d8-6ff6-431f-80d7-c4fcd005d522 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:11.509443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df89d8-6ff6-431f-80d7-c4fcd005d522 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:11.509443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df89d8-6ff6-431f-80d7-c4fcd005d522 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:11.510719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 554/1099] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:11.517770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550f8f2-9c38-4b6d-b61a-2573caafada5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:11.518225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550f8f2-9c38-4b6d-b61a-2573caafada5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:11.518321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550f8f2-9c38-4b6d-b61a-2573caafada5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:11.538209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1550f8f2-9c38-4b6d-b61a-2573caafada5 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:11.539402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1550f8f2-9c38-4b6d-b61a-2573caafada5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:11.608740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 546/1100] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:42:11.632886 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df89d8-6ff6-431f-80d7-c4fcd005d522 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:11.636749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:11.737398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df89d8-6ff6-431f-80d7-c4fcd005d522 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:11.771219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aef841-cd25-466f-8a7e-8344ce0f7135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:11.771762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aef841-cd25-466f-8a7e-8344ce0f7135 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:11.772041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aef841-cd25-466f-8a7e-8344ce0f7135 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:11.772312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aef841-cd25-466f-8a7e-8344ce0f7135 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:11.782014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aef841-cd25-466f-8a7e-8344ce0f7135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-49aef841-cd25-466f-8a7e-8344ce0f7135 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:11.786716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:11.787819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:11.788862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 555/1101] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:11.793913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550f8f2-9c38-4b6d-b61a-2573caafada5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:11.880292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:11.891736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550f8f2-9c38-4b6d-b61a-2573caafada5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:11.925772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee6d8b-a160-4e4b-aa2e-ba967521fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:11.926380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6d8b-a160-4e4b-aa2e-ba967521fa1a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:11.926660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6d8b-a160-4e4b-aa2e-ba967521fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:11.926933 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee6d8b-a160-4e4b-aa2e-ba967521fa1a None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:11.927595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee6d8b-a160-4e4b-aa2e-ba967521fa1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:11.927896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee6d8b-a160-4e4b-aa2e-ba967521fa1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:11.929281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee6d8b-a160-4e4b-aa2e-ba967521fa1a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:11.937202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 547/1102] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:11.943963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aef841-cd25-466f-8a7e-8344ce0f7135 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:12.006377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:12.043624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aef841-cd25-466f-8a7e-8344ce0f7135 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:12.077148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4232ecd-cb95-4ae5-8e1d-14ae03ca25db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:12.077766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4232ecd-cb95-4ae5-8e1d-14ae03ca25db None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:12.078046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4232ecd-cb95-4ae5-8e1d-14ae03ca25db None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:12.078318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4232ecd-cb95-4ae5-8e1d-14ae03ca25db None admin] PATH_INFO: `/v3/users/864f9e99afdc41b0a21c7914460b2a4d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:12.079420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4232ecd-cb95-4ae5-8e1d-14ae03ca25db None admin] RBAC: Authorizing `identity:get_user(user_id=864f9e99afdc41b0a21c7914460b2a4d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:12.081758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4232ecd-cb95-4ae5-8e1d-14ae03ca25db None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:12.083348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 556/1103] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:11 2022] GET /identity/v3/users/864f9e99afdc41b0a21c7914460b2a4d => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:12.089293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee6d8b-a160-4e4b-aa2e-ba967521fa1a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:12.185440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee6d8b-a160-4e4b-aa2e-ba967521fa1a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:12.219252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaa766-580d-4078-9336-156c918ca961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:12.219858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaa766-580d-4078-9336-156c918ca961 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:12.220305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaa766-580d-4078-9336-156c918ca961 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:12.220460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaa766-580d-4078-9336-156c918ca961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:12.238108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:12.286075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deaa766-580d-4078-9336-156c918ca961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:12.287483 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deaa766-580d-4078-9336-156c918ca961 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:12.381514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 548/1104] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:42:12.387284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4232ecd-cb95-4ae5-8e1d-14ae03ca25db None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:12.484819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4232ecd-cb95-4ae5-8e1d-14ae03ca25db None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:12.512753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:12.518184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27374da1-5710-40a7-9a79-48f185f46bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:12.518817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27374da1-5710-40a7-9a79-48f185f46bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:12.519095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27374da1-5710-40a7-9a79-48f185f46bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:12.519364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27374da1-5710-40a7-9a79-48f185f46bf8 None admin] PATH_INFO: `/v3/projects/fb81afa93d6f4924b0d150937d380722` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:12.520474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27374da1-5710-40a7-9a79-48f185f46bf8 None admin] RBAC: Authorizing `identity:get_project(project_id=fb81afa93d6f4924b0d150937d380722)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:12.521920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27374da1-5710-40a7-9a79-48f185f46bf8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:12.523517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 557/1105] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:12 2022] GET /identity/v3/projects/fb81afa93d6f4924b0d150937d380722 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:12.529580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaa766-580d-4078-9336-156c918ca961 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:12.577504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:12.626402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaa766-580d-4078-9336-156c918ca961 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:12.659753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcaa36-64bd-4dbb-94de-b69e9c621c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:12.660288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcaa36-64bd-4dbb-94de-b69e9c621c26 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:12.660567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcaa36-64bd-4dbb-94de-b69e9c621c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:12.660838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcaa36-64bd-4dbb-94de-b69e9c621c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:12.661597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcaa36-64bd-4dbb-94de-b69e9c621c26 None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=fb81afa93d6f4924b0d150937d380722, user.id=864f9e99afdc41b0a21c7914460b2a4d) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:12.661908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcaa36-64bd-4dbb-94de-b69e9c621c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:12.663312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcaa36-64bd-4dbb-94de-b69e9c621c26 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:12.668895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 549/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:42:12 2022] GET /identity/v3/role_assignments?user.id=864f9e99afdc41b0a21c7914460b2a4d&scope.project.id=fb81afa93d6f4924b0d150937d380722&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:14.676152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc04147-e41b-43e2-b4e3-ee1758d7c93b None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:14.676686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc04147-e41b-43e2-b4e3-ee1758d7c93b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:14.676686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc04147-e41b-43e2-b4e3-ee1758d7c93b None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:14.678041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 558/1107] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:14.683244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bca46b-46be-4183-acac-f00fc9765d8e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:14.683647 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bca46b-46be-4183-acac-f00fc9765d8e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:14.683792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bca46b-46be-4183-acac-f00fc9765d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:14.701824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c3bca46b-46be-4183-acac-f00fc9765d8e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:14.703005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c3bca46b-46be-4183-acac-f00fc9765d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:14.769363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 550/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:14.793512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc04147-e41b-43e2-b4e3-ee1758d7c93b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:14.888840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc04147-e41b-43e2-b4e3-ee1758d7c93b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:14.921518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1fa2d-45cb-4658-8113-4077012aea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:14.922121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1fa2d-45cb-4658-8113-4077012aea00 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:14.922394 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1fa2d-45cb-4658-8113-4077012aea00 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:14.922660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1fa2d-45cb-4658-8113-4077012aea00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:14.923863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1fa2d-45cb-4658-8113-4077012aea00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:14.925302 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1fa2d-45cb-4658-8113-4077012aea00 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:14.926846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 559/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:14.932358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bca46b-46be-4183-acac-f00fc9765d8e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:14.976787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:15.028913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bca46b-46be-4183-acac-f00fc9765d8e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:15.060837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52df5d-b898-47b9-9b9a-f5e5d0f0e5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:15.061466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52df5d-b898-47b9-9b9a-f5e5d0f0e5ef None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:15.061745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52df5d-b898-47b9-9b9a-f5e5d0f0e5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:15.062017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52df5d-b898-47b9-9b9a-f5e5d0f0e5ef None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:15.062651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52df5d-b898-47b9-9b9a-f5e5d0f0e5ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:15.064613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52df5d-b898-47b9-9b9a-f5e5d0f0e5ef None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:15.147776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 551/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:42:14 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:42:17.158980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92e123-5724-44f2-b09b-9e039edefe55 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:17.159513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92e123-5724-44f2-b09b-9e039edefe55 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:17.159513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92e123-5724-44f2-b09b-9e039edefe55 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:17.160813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 560/1111] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:17.166141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120da91a-789b-4700-ad7a-2542bc34d7a1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:17.166510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120da91a-789b-4700-ad7a-2542bc34d7a1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:17.166699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120da91a-789b-4700-ad7a-2542bc34d7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:17.189439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-120da91a-789b-4700-ad7a-2542bc34d7a1 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:17.190676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-120da91a-789b-4700-ad7a-2542bc34d7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:17.266550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 552/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:17.290028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92e123-5724-44f2-b09b-9e039edefe55 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:17.356875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:17.390464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92e123-5724-44f2-b09b-9e039edefe55 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:17.424824 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab438d7-16f5-498a-b1ba-32a5a2ec6af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:17.425415 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab438d7-16f5-498a-b1ba-32a5a2ec6af9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:17.425696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab438d7-16f5-498a-b1ba-32a5a2ec6af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:17.425968 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab438d7-16f5-498a-b1ba-32a5a2ec6af9 None admin] PATH_INFO: `/v3/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:17.427371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab438d7-16f5-498a-b1ba-32a5a2ec6af9 None admin] RBAC: Authorizing `identity:get_role(role_id=cfdf14ad1f2e4a7d9eeed35bc4ffa596)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:17.428729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab438d7-16f5-498a-b1ba-32a5a2ec6af9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:17.430077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 561/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:17 2022] GET /identity/v3/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:17.435905 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120da91a-789b-4700-ad7a-2542bc34d7a1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:17.466218 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:17.534487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120da91a-789b-4700-ad7a-2542bc34d7a1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:17.568105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f53c80-7dc2-4945-99f8-c0f0dba7202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:17.568733 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53c80-7dc2-4945-99f8-c0f0dba7202d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:17.569037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53c80-7dc2-4945-99f8-c0f0dba7202d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:17.569310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53c80-7dc2-4945-99f8-c0f0dba7202d None admin] PATH_INFO: `/v3/users/936336057334407b818ffcd1b722ec16` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:17.582036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f53c80-7dc2-4945-99f8-c0f0dba7202d None admin] RBAC: Authorizing `identity:get_user(user_id=936336057334407b818ffcd1b722ec16)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:17.583456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f53c80-7dc2-4945-99f8-c0f0dba7202d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:17.585043 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 553/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:17 2022] GET /identity/v3/users/936336057334407b818ffcd1b722ec16 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:17.590866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab438d7-16f5-498a-b1ba-32a5a2ec6af9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:17.691569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab438d7-16f5-498a-b1ba-32a5a2ec6af9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:17.724145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858dae3-52c8-45ab-8a0b-358ab2746ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:17.724751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858dae3-52c8-45ab-8a0b-358ab2746ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:17.725050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858dae3-52c8-45ab-8a0b-358ab2746ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:17.725332 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858dae3-52c8-45ab-8a0b-358ab2746ad7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:17.728760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:17.789771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858dae3-52c8-45ab-8a0b-358ab2746ad7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:17.791199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858dae3-52c8-45ab-8a0b-358ab2746ad7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:17.886187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 562/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:42:17.892215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f53c80-7dc2-4945-99f8-c0f0dba7202d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:17.977458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:17.989592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f53c80-7dc2-4945-99f8-c0f0dba7202d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:18.022660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb232382-2d01-4a72-8a1a-252c917e3193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:18.023287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb232382-2d01-4a72-8a1a-252c917e3193 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:18.023571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb232382-2d01-4a72-8a1a-252c917e3193 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:18.023850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb232382-2d01-4a72-8a1a-252c917e3193 None admin] PATH_INFO: `/v3/projects/fb81afa93d6f4924b0d150937d380722` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:18.024961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb232382-2d01-4a72-8a1a-252c917e3193 None admin] RBAC: Authorizing `identity:get_project(project_id=fb81afa93d6f4924b0d150937d380722)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:18.026393 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb232382-2d01-4a72-8a1a-252c917e3193 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:18.027990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 554/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:17 2022] GET /identity/v3/projects/fb81afa93d6f4924b0d150937d380722 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:18.034800 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858dae3-52c8-45ab-8a0b-358ab2746ad7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:18.068374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:18.130969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858dae3-52c8-45ab-8a0b-358ab2746ad7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:18.163252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157610ca-325d-4bd9-9c68-7e2103a56f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:18.163845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157610ca-325d-4bd9-9c68-7e2103a56f78 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:18.164094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157610ca-325d-4bd9-9c68-7e2103a56f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:18.164361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157610ca-325d-4bd9-9c68-7e2103a56f78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:18.165092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157610ca-325d-4bd9-9c68-7e2103a56f78 None admin] RBAC: Adding query filter params (role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596, scope.project.id=fb81afa93d6f4924b0d150937d380722, user.id=936336057334407b818ffcd1b722ec16) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:18.165393 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157610ca-325d-4bd9-9c68-7e2103a56f78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:18.166786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157610ca-325d-4bd9-9c68-7e2103a56f78 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:18.171858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 563/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:42:18 2022] GET /identity/v3/role_assignments?user.id=936336057334407b818ffcd1b722ec16&scope.project.id=fb81afa93d6f4924b0d150937d380722&role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:20.139483 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07625071-2699-4b26-a80c-2108acae4df2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:20.140036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07625071-2699-4b26-a80c-2108acae4df2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:20.140036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07625071-2699-4b26-a80c-2108acae4df2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:20.141376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 555/1118] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:20.147130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37372c-e2e7-4831-84a8-388f730f2bc0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:20.147523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37372c-e2e7-4831-84a8-388f730f2bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:20.147678 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37372c-e2e7-4831-84a8-388f730f2bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:20.165710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-bc37372c-e2e7-4831-84a8-388f730f2bc0 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:20.166904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bc37372c-e2e7-4831-84a8-388f730f2bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:20.231068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 564/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:20.253637 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07625071-2699-4b26-a80c-2108acae4df2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:20.351467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07625071-2699-4b26-a80c-2108acae4df2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:20.384318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16856956-cfc6-49a6-bb31-05e68fc36f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:20.384934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16856956-cfc6-49a6-bb31-05e68fc36f09 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:20.385243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16856956-cfc6-49a6-bb31-05e68fc36f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:20.385512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16856956-cfc6-49a6-bb31-05e68fc36f09 None admin] PATH_INFO: `/v3/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:20.386875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16856956-cfc6-49a6-bb31-05e68fc36f09 None admin] RBAC: Authorizing `identity:get_role(role_id=cfdf14ad1f2e4a7d9eeed35bc4ffa596)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:20.388199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16856956-cfc6-49a6-bb31-05e68fc36f09 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:20.389575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 556/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:20 2022] GET /identity/v3/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:20.395651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37372c-e2e7-4831-84a8-388f730f2bc0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:20.423452 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:20.488717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37372c-e2e7-4831-84a8-388f730f2bc0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:20.520600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc906d41-00ad-44cf-b6ac-7e14452c63cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:20.521223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc906d41-00ad-44cf-b6ac-7e14452c63cd None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:20.521500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc906d41-00ad-44cf-b6ac-7e14452c63cd None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:20.521768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc906d41-00ad-44cf-b6ac-7e14452c63cd None admin] PATH_INFO: `/v3/users/936336057334407b818ffcd1b722ec16` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:20.522861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc906d41-00ad-44cf-b6ac-7e14452c63cd None admin] RBAC: Authorizing `identity:get_user(user_id=936336057334407b818ffcd1b722ec16)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:20.524268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc906d41-00ad-44cf-b6ac-7e14452c63cd None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:20.525818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 565/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:20 2022] GET /identity/v3/users/936336057334407b818ffcd1b722ec16 => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:20.531720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16856956-cfc6-49a6-bb31-05e68fc36f09 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:20.556601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:20.631314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16856956-cfc6-49a6-bb31-05e68fc36f09 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:20.664874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76734699-542c-46da-a700-fcb9d23baf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:20.665467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76734699-542c-46da-a700-fcb9d23baf88 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:20.665748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76734699-542c-46da-a700-fcb9d23baf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:20.666020 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76734699-542c-46da-a700-fcb9d23baf88 None admin] PATH_INFO: `/v3/projects/fb81afa93d6f4924b0d150937d380722` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:20.667143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76734699-542c-46da-a700-fcb9d23baf88 None admin] RBAC: Authorizing `identity:get_project(project_id=fb81afa93d6f4924b0d150937d380722)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:20.668554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76734699-542c-46da-a700-fcb9d23baf88 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:20.670218 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 557/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:20 2022] GET /identity/v3/projects/fb81afa93d6f4924b0d150937d380722 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:20.676875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc906d41-00ad-44cf-b6ac-7e14452c63cd None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:20.769167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:20.772894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc906d41-00ad-44cf-b6ac-7e14452c63cd None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:20.806087 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148cf44-9652-41f6-9778-30b1dcb4c085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:20.806738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148cf44-9652-41f6-9778-30b1dcb4c085 None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:20.807008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148cf44-9652-41f6-9778-30b1dcb4c085 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:20.807280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148cf44-9652-41f6-9778-30b1dcb4c085 None admin] PATH_INFO: `/v3/projects/fb81afa93d6f4924b0d150937d380722/users/936336057334407b818ffcd1b722ec16/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:20.809423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148cf44-9652-41f6-9778-30b1dcb4c085 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb81afa93d6f4924b0d150937d380722, user_id=936336057334407b818ffcd1b722ec16, role_id=cfdf14ad1f2e4a7d9eeed35bc4ffa596)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:20.810848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148cf44-9652-41f6-9778-30b1dcb4c085 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:20.872698 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 566/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:42:20 2022] PUT /identity/v3/projects/fb81afa93d6f4924b0d150937d380722/users/936336057334407b818ffcd1b722ec16/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:42:22.838424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c15a62-fc45-4eb2-a6ad-da21d4bc0af3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:22.838953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c15a62-fc45-4eb2-a6ad-da21d4bc0af3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:22.838953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c15a62-fc45-4eb2-a6ad-da21d4bc0af3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:22.840295 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 558/1124] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:22.846331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c419b-f89e-4395-ba69-e26d3a352d68 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:22.846733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c419b-f89e-4395-ba69-e26d3a352d68 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:22.846867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c419b-f89e-4395-ba69-e26d3a352d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:22.864930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-af3c419b-f89e-4395-ba69-e26d3a352d68 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:22.866146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-af3c419b-f89e-4395-ba69-e26d3a352d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:22.931983 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 567/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:22.955317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c15a62-fc45-4eb2-a6ad-da21d4bc0af3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:23.043765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:23.053505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c15a62-fc45-4eb2-a6ad-da21d4bc0af3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:23.086365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c56fd7-ea70-4124-b51d-16ac415a63a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:23.086976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c56fd7-ea70-4124-b51d-16ac415a63a7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:23.087255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c56fd7-ea70-4124-b51d-16ac415a63a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:23.087527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c56fd7-ea70-4124-b51d-16ac415a63a7 None admin] PATH_INFO: `/v3/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:23.088956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c56fd7-ea70-4124-b51d-16ac415a63a7 None admin] RBAC: Authorizing `identity:get_role(role_id=cfdf14ad1f2e4a7d9eeed35bc4ffa596)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:23.090276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c56fd7-ea70-4124-b51d-16ac415a63a7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:23.091560 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 559/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:22 2022] GET /identity/v3/roles/cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:23.097687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c419b-f89e-4395-ba69-e26d3a352d68 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:23.181751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:23.193916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c419b-f89e-4395-ba69-e26d3a352d68 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:23.228240 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be3245-4026-4028-a304-bcc080c97e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:23.228860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be3245-4026-4028-a304-bcc080c97e79 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:23.229175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be3245-4026-4028-a304-bcc080c97e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:23.229452 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be3245-4026-4028-a304-bcc080c97e79 None admin] PATH_INFO: `/v3/users/936336057334407b818ffcd1b722ec16` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:23.230560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be3245-4026-4028-a304-bcc080c97e79 None admin] RBAC: Authorizing `identity:get_user(user_id=936336057334407b818ffcd1b722ec16)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:23.231993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be3245-4026-4028-a304-bcc080c97e79 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:23.233776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 568/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:23 2022] GET /identity/v3/users/936336057334407b818ffcd1b722ec16 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:23.240224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c56fd7-ea70-4124-b51d-16ac415a63a7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:23.335566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c56fd7-ea70-4124-b51d-16ac415a63a7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:23.368658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a0c44-28e8-4f9b-b1b3-22b1a90d4b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:23.369227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a0c44-28e8-4f9b-b1b3-22b1a90d4b94 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:23.369510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a0c44-28e8-4f9b-b1b3-22b1a90d4b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:23.369780 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a0c44-28e8-4f9b-b1b3-22b1a90d4b94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:23.395600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:23.434998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a0c44-28e8-4f9b-b1b3-22b1a90d4b94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:23.436415 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a0c44-28e8-4f9b-b1b3-22b1a90d4b94 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:23.529437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 560/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:42:23.535643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be3245-4026-4028-a304-bcc080c97e79 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:23.632318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be3245-4026-4028-a304-bcc080c97e79 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:23.664332 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77763b3f-f7dc-4e3b-be57-eb555ae81523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:23.664943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77763b3f-f7dc-4e3b-be57-eb555ae81523 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:23.665247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77763b3f-f7dc-4e3b-be57-eb555ae81523 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:23.665521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77763b3f-f7dc-4e3b-be57-eb555ae81523 None admin] PATH_INFO: `/v3/projects/fb81afa93d6f4924b0d150937d380722` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:23.666628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77763b3f-f7dc-4e3b-be57-eb555ae81523 None admin] RBAC: Authorizing `identity:get_project(project_id=fb81afa93d6f4924b0d150937d380722)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:23.668033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77763b3f-f7dc-4e3b-be57-eb555ae81523 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:23.669647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 569/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:23 2022] GET /identity/v3/projects/fb81afa93d6f4924b0d150937d380722 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:23.676190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a0c44-28e8-4f9b-b1b3-22b1a90d4b94 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:23.733164 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:23.772435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a0c44-28e8-4f9b-b1b3-22b1a90d4b94 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:23.805231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8f3b0-e6e4-4e9b-83bf-297ef3eba698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:23.805827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8f3b0-e6e4-4e9b-83bf-297ef3eba698 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:23.806105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8f3b0-e6e4-4e9b-83bf-297ef3eba698 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:23.806377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8f3b0-e6e4-4e9b-83bf-297ef3eba698 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:23.807095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8f3b0-e6e4-4e9b-83bf-297ef3eba698 None admin] RBAC: Adding query filter params (role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596, scope.project.id=fb81afa93d6f4924b0d150937d380722, user.id=936336057334407b818ffcd1b722ec16) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:23.807403 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8f3b0-e6e4-4e9b-83bf-297ef3eba698 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:23.808797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8f3b0-e6e4-4e9b-83bf-297ef3eba698 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:23.815032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 561/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:42:23 2022] GET /identity/v3/role_assignments?user.id=936336057334407b818ffcd1b722ec16&scope.project.id=fb81afa93d6f4924b0d150937d380722&role.id=cfdf14ad1f2e4a7d9eeed35bc4ffa596 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:25.825195 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8370-9567-4557-9b09-d9c53f8f43d0 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:25.825773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8370-9567-4557-9b09-d9c53f8f43d0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:25.825773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8370-9567-4557-9b09-d9c53f8f43d0 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:25.827045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 570/1131] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:25.832387 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85947e17-080a-48ce-a2f0-b4e0a92a0249 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:25.832805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85947e17-080a-48ce-a2f0-b4e0a92a0249 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:25.832929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85947e17-080a-48ce-a2f0-b4e0a92a0249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:25.851629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-85947e17-080a-48ce-a2f0-b4e0a92a0249 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:25.852829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-85947e17-080a-48ce-a2f0-b4e0a92a0249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:25.917935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 562/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:25.942039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb8370-9567-4557-9b09-d9c53f8f43d0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:25.967198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:26.042571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb8370-9567-4557-9b09-d9c53f8f43d0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:26.075650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1794d-ef1f-4ba4-a896-96f823bb0eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:26.076203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1794d-ef1f-4ba4-a896-96f823bb0eef None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:26.076481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1794d-ef1f-4ba4-a896-96f823bb0eef None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:26.076754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1794d-ef1f-4ba4-a896-96f823bb0eef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:26.077883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1794d-ef1f-4ba4-a896-96f823bb0eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:26.080021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1794d-ef1f-4ba4-a896-96f823bb0eef None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:26.081590 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 571/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:26.087387 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85947e17-080a-48ce-a2f0-b4e0a92a0249 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:26.140425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:26.185182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85947e17-080a-48ce-a2f0-b4e0a92a0249 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:26.218882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67566c3d-6df8-4568-a0d1-fd6814323656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:26.219506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67566c3d-6df8-4568-a0d1-fd6814323656 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:26.219795 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67566c3d-6df8-4568-a0d1-fd6814323656 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:26.220071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67566c3d-6df8-4568-a0d1-fd6814323656 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:26.220881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67566c3d-6df8-4568-a0d1-fd6814323656 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:26.222338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67566c3d-6df8-4568-a0d1-fd6814323656 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:26.331088 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 563/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:42:26 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:42:28.325435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e4a3-3457-4695-bc08-2c1975da7138 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:28.325962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e4a3-3457-4695-bc08-2c1975da7138 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:28.326070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e4a3-3457-4695-bc08-2c1975da7138 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:28.327290 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 572/1135] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:28.333135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5153b7-c1c1-47ac-93d7-ebfd4cabbf14 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:28.333555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5153b7-c1c1-47ac-93d7-ebfd4cabbf14 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:28.333683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5153b7-c1c1-47ac-93d7-ebfd4cabbf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:28.351605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2e5153b7-c1c1-47ac-93d7-ebfd4cabbf14 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:28.352779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2e5153b7-c1c1-47ac-93d7-ebfd4cabbf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:28.416995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 564/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:42:28.441659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7e4a3-3457-4695-bc08-2c1975da7138 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:28.530870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:28.539931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7e4a3-3457-4695-bc08-2c1975da7138 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:28.573206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5908c1-5333-4897-ba02-b5d0a5e8a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:28.573822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5908c1-5333-4897-ba02-b5d0a5e8a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:28.574100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5908c1-5333-4897-ba02-b5d0a5e8a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:28.574390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5908c1-5333-4897-ba02-b5d0a5e8a8c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:28.575475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5908c1-5333-4897-ba02-b5d0a5e8a8c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:28.576892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5908c1-5333-4897-ba02-b5d0a5e8a8c7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:28.578461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 573/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:42:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:28.584335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5153b7-c1c1-47ac-93d7-ebfd4cabbf14 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:28.617999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:28.680316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5153b7-c1c1-47ac-93d7-ebfd4cabbf14 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:28.712207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6c7a6-bff1-4c70-8650-93e4b0fa4a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:28.712741 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6c7a6-bff1-4c70-8650-93e4b0fa4a6e None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:28.713040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6c7a6-bff1-4c70-8650-93e4b0fa4a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:28.713316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6c7a6-bff1-4c70-8650-93e4b0fa4a6e None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:28.713947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6c7a6-bff1-4c70-8650-93e4b0fa4a6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:28.715342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6c7a6-bff1-4c70-8650-93e4b0fa4a6e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:28.809700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 565/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:42:28 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:42:30.815965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb8c2-71ec-481a-90f3-935488fc678c None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:30.816488 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb8c2-71ec-481a-90f3-935488fc678c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:30.816488 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb8c2-71ec-481a-90f3-935488fc678c None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:30.818103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 574/1139] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:30.823282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc3d59-8864-4bb9-9597-e2c587662213 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:30.823668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc3d59-8864-4bb9-9597-e2c587662213 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:30.823828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fc3d59-8864-4bb9-9597-e2c587662213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:30.842218 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c3fc3d59-8864-4bb9-9597-e2c587662213 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:30.843404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c3fc3d59-8864-4bb9-9597-e2c587662213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:30.906976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 566/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 09:42:30.929682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb8c2-71ec-481a-90f3-935488fc678c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:31.030545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb8c2-71ec-481a-90f3-935488fc678c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:31.069184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a575223-4221-4645-934a-cd5a9ff092c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:31.070057 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a575223-4221-4645-934a-cd5a9ff092c3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:31.070338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a575223-4221-4645-934a-cd5a9ff092c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:31.070611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a575223-4221-4645-934a-cd5a9ff092c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:31.080579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a575223-4221-4645-934a-cd5a9ff092c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-6a575223-4221-4645-934a-cd5a9ff092c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:31.085324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:31.086464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:31.087371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 575/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:42:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:31.092663 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc3d59-8864-4bb9-9597-e2c587662213 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:31.116490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:31.191660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fc3d59-8864-4bb9-9597-e2c587662213 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:31.226594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8ac95-f60f-406c-8c84-0bf18fe16c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:31.227195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8ac95-f60f-406c-8c84-0bf18fe16c86 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:31.227478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8ac95-f60f-406c-8c84-0bf18fe16c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:31.227753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8ac95-f60f-406c-8c84-0bf18fe16c86 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:31.228414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8ac95-f60f-406c-8c84-0bf18fe16c86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:31.228720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8ac95-f60f-406c-8c84-0bf18fe16c86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:31.230101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8ac95-f60f-406c-8c84-0bf18fe16c86 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:31.238517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 567/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:42:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:31.244697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a575223-4221-4645-934a-cd5a9ff092c3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:31.267450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:31.343687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a575223-4221-4645-934a-cd5a9ff092c3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:31.377149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d94bb3-b132-4dff-b24f-f91477ff29e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:31.377764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d94bb3-b132-4dff-b24f-f91477ff29e3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:31.378044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d94bb3-b132-4dff-b24f-f91477ff29e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:31.378316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d94bb3-b132-4dff-b24f-f91477ff29e3 None admin] PATH_INFO: `/v3/users/31e63320afd4449aacdbbf9a108a3a95` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:31.390976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d94bb3-b132-4dff-b24f-f91477ff29e3 None admin] RBAC: Authorizing `identity:get_user(user_id=31e63320afd4449aacdbbf9a108a3a95)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:31.393049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d94bb3-b132-4dff-b24f-f91477ff29e3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:31.394628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 576/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:31 2022] GET /identity/v3/users/31e63320afd4449aacdbbf9a108a3a95 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:31.400366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8ac95-f60f-406c-8c84-0bf18fe16c86 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:31.483344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:31.498959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8ac95-f60f-406c-8c84-0bf18fe16c86 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:31.532926 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f8e33-9df3-4324-abcc-48490c894691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:31.533555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f8e33-9df3-4324-abcc-48490c894691 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:31.533835 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f8e33-9df3-4324-abcc-48490c894691 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:31.534106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f8e33-9df3-4324-abcc-48490c894691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:31.600514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f8e33-9df3-4324-abcc-48490c894691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:31.601906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f8e33-9df3-4324-abcc-48490c894691 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:31.682407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:31.698177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 568/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:42:31.704415 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d94bb3-b132-4dff-b24f-f91477ff29e3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:31.782582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:31.802880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d94bb3-b132-4dff-b24f-f91477ff29e3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:31.836302 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd861b1c-2641-48c2-a8bb-363db90715d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:31.836901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd861b1c-2641-48c2-a8bb-363db90715d7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:31.837245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd861b1c-2641-48c2-a8bb-363db90715d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:31.837518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd861b1c-2641-48c2-a8bb-363db90715d7 None admin] PATH_INFO: `/v3/projects/f61d62eeb7664708bf1d53e845c69ffb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:31.838620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd861b1c-2641-48c2-a8bb-363db90715d7 None admin] RBAC: Authorizing `identity:get_project(project_id=f61d62eeb7664708bf1d53e845c69ffb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:31.840033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd861b1c-2641-48c2-a8bb-363db90715d7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:31.841694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 577/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:31 2022] GET /identity/v3/projects/f61d62eeb7664708bf1d53e845c69ffb => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:31.848674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f8e33-9df3-4324-abcc-48490c894691 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:31.945881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f8e33-9df3-4324-abcc-48490c894691 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:31.979880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a410bbe-fe4e-4589-9d35-e1edffab0026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:31.980489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410bbe-fe4e-4589-9d35-e1edffab0026 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:31.980766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410bbe-fe4e-4589-9d35-e1edffab0026 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:31.981059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410bbe-fe4e-4589-9d35-e1edffab0026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:31.981783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a410bbe-fe4e-4589-9d35-e1edffab0026 None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=f61d62eeb7664708bf1d53e845c69ffb, user.id=31e63320afd4449aacdbbf9a108a3a95) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:31.982086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a410bbe-fe4e-4589-9d35-e1edffab0026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:31.983481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a410bbe-fe4e-4589-9d35-e1edffab0026 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:31.988840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 569/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:42:31 2022] GET /identity/v3/role_assignments?user.id=31e63320afd4449aacdbbf9a108a3a95&scope.project.id=f61d62eeb7664708bf1d53e845c69ffb&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:33.939264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d5290-5cbd-41c8-83ea-6c22af5da586 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:33.939819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d5290-5cbd-41c8-83ea-6c22af5da586 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:33.939819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d5290-5cbd-41c8-83ea-6c22af5da586 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:33.941113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 578/1147] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:33.946327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129fa38-2f16-47f6-a434-67eb0812a82e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:33.946761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129fa38-2f16-47f6-a434-67eb0812a82e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:33.946889 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129fa38-2f16-47f6-a434-67eb0812a82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:33.965220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2129fa38-2f16-47f6-a434-67eb0812a82e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:33.966407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2129fa38-2f16-47f6-a434-67eb0812a82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:34.008495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:34.030949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 570/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:34.053267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d5290-5cbd-41c8-83ea-6c22af5da586 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:34.149901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d5290-5cbd-41c8-83ea-6c22af5da586 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:34.181963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be1608-6f34-4ee5-a6d7-ac4ef13a1a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:34.182582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be1608-6f34-4ee5-a6d7-ac4ef13a1a5d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:34.182858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be1608-6f34-4ee5-a6d7-ac4ef13a1a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:34.183127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be1608-6f34-4ee5-a6d7-ac4ef13a1a5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:34.194116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be1608-6f34-4ee5-a6d7-ac4ef13a1a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-34be1608-6f34-4ee5-a6d7-ac4ef13a1a5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:34.198664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:34.199759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:34.200845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 579/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:42:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:34.208000 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129fa38-2f16-47f6-a434-67eb0812a82e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:34.303996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129fa38-2f16-47f6-a434-67eb0812a82e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:34.338097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddef25-d0fd-4574-a7f0-a0e4a3f8fe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:34.338688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddef25-d0fd-4574-a7f0-a0e4a3f8fe72 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:34.338967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddef25-d0fd-4574-a7f0-a0e4a3f8fe72 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:34.339238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddef25-d0fd-4574-a7f0-a0e4a3f8fe72 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:34.339894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddef25-d0fd-4574-a7f0-a0e4a3f8fe72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:34.340192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddef25-d0fd-4574-a7f0-a0e4a3f8fe72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:34.341568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddef25-d0fd-4574-a7f0-a0e4a3f8fe72 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:34.349497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 571/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:42:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:34.355225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be1608-6f34-4ee5-a6d7-ac4ef13a1a5d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:34.364645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:34.452177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be1608-6f34-4ee5-a6d7-ac4ef13a1a5d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:34.484091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec484c85-d242-44db-aa66-30a1410e66f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:34.484707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec484c85-d242-44db-aa66-30a1410e66f4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:34.485582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec484c85-d242-44db-aa66-30a1410e66f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:34.485938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec484c85-d242-44db-aa66-30a1410e66f4 None admin] PATH_INFO: `/v3/users/31e63320afd4449aacdbbf9a108a3a95` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:34.487049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec484c85-d242-44db-aa66-30a1410e66f4 None admin] RBAC: Authorizing `identity:get_user(user_id=31e63320afd4449aacdbbf9a108a3a95)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:34.488453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec484c85-d242-44db-aa66-30a1410e66f4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:34.490020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 580/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:34 2022] GET /identity/v3/users/31e63320afd4449aacdbbf9a108a3a95 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:34.495645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddef25-d0fd-4574-a7f0-a0e4a3f8fe72 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:34.536280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:34.591387 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddef25-d0fd-4574-a7f0-a0e4a3f8fe72 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:34.623637 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b81a7-ed81-465b-85f9-d42b6865fdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:34.624329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b81a7-ed81-465b-85f9-d42b6865fdb0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:34.624611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b81a7-ed81-465b-85f9-d42b6865fdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:34.624882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b81a7-ed81-465b-85f9-d42b6865fdb0 None admin] PATH_INFO: `/v3/projects/f61d62eeb7664708bf1d53e845c69ffb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:34.626017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b81a7-ed81-465b-85f9-d42b6865fdb0 None admin] RBAC: Authorizing `identity:get_project(project_id=f61d62eeb7664708bf1d53e845c69ffb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:34.627401 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b81a7-ed81-465b-85f9-d42b6865fdb0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:34.628980 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 572/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:34 2022] GET /identity/v3/projects/f61d62eeb7664708bf1d53e845c69ffb => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:34.634768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec484c85-d242-44db-aa66-30a1410e66f4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:34.707852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:34.731636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec484c85-d242-44db-aa66-30a1410e66f4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:34.764918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ca1c2-1d7c-4e8f-800f-b425314fc82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:34.765575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ca1c2-1d7c-4e8f-800f-b425314fc82f None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:34.765852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ca1c2-1d7c-4e8f-800f-b425314fc82f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:34.766636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ca1c2-1d7c-4e8f-800f-b425314fc82f None admin] PATH_INFO: `/v3/projects/f61d62eeb7664708bf1d53e845c69ffb/users/31e63320afd4449aacdbbf9a108a3a95/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:34.768497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ca1c2-1d7c-4e8f-800f-b425314fc82f None admin] RBAC: Authorizing `identity:create_grant(project_id=f61d62eeb7664708bf1d53e845c69ffb, user_id=31e63320afd4449aacdbbf9a108a3a95, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:34.769935 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ca1c2-1d7c-4e8f-800f-b425314fc82f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:34.855015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 581/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:42:34 2022] PUT /identity/v3/projects/f61d62eeb7664708bf1d53e845c69ffb/users/31e63320afd4449aacdbbf9a108a3a95/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:42:36.831111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a1880-cd4f-416d-83dc-6e86ad7266f6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:36.831661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a1880-cd4f-416d-83dc-6e86ad7266f6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:36.831661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a1880-cd4f-416d-83dc-6e86ad7266f6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:36.832990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 573/1154] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:36.838144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3f74a-4b22-4f4c-8c83-690f67d371d9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:36.838580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3f74a-4b22-4f4c-8c83-690f67d371d9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:36.838689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3f74a-4b22-4f4c-8c83-690f67d371d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:36.856739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c1d3f74a-4b22-4f4c-8c83-690f67d371d9 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:36.857974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c1d3f74a-4b22-4f4c-8c83-690f67d371d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:36.923110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 582/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:36.945909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a1880-cd4f-416d-83dc-6e86ad7266f6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:37.045652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a1880-cd4f-416d-83dc-6e86ad7266f6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:37.054684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:37.078184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034a8e7-c123-4b1d-959b-918d9da2d053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:37.082930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034a8e7-c123-4b1d-959b-918d9da2d053 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:37.082930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034a8e7-c123-4b1d-959b-918d9da2d053 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:37.082930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034a8e7-c123-4b1d-959b-918d9da2d053 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:37.089565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034a8e7-c123-4b1d-959b-918d9da2d053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f034a8e7-c123-4b1d-959b-918d9da2d053 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:37.094771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:37.095864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:37.096858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 574/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:42:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:37.101931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3f74a-4b22-4f4c-8c83-690f67d371d9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:37.160789 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:37.196638 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3f74a-4b22-4f4c-8c83-690f67d371d9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:37.231397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b068b32-fdf8-481c-b7b0-cba3a36760af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:37.231989 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b068b32-fdf8-481c-b7b0-cba3a36760af None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:37.232267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b068b32-fdf8-481c-b7b0-cba3a36760af None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:37.232538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b068b32-fdf8-481c-b7b0-cba3a36760af None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:37.233225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b068b32-fdf8-481c-b7b0-cba3a36760af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:37.233527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b068b32-fdf8-481c-b7b0-cba3a36760af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:37.234977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b068b32-fdf8-481c-b7b0-cba3a36760af None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:37.242632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 583/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:42:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:37.248404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034a8e7-c123-4b1d-959b-918d9da2d053 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:37.346194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034a8e7-c123-4b1d-959b-918d9da2d053 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:37.378607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600119f-9fc7-491a-a09f-b6376ab63dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:37.379217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600119f-9fc7-491a-a09f-b6376ab63dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:37.379494 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600119f-9fc7-491a-a09f-b6376ab63dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:37.379765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600119f-9fc7-491a-a09f-b6376ab63dd6 None admin] PATH_INFO: `/v3/users/31e63320afd4449aacdbbf9a108a3a95` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:37.380867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600119f-9fc7-491a-a09f-b6376ab63dd6 None admin] RBAC: Authorizing `identity:get_user(user_id=31e63320afd4449aacdbbf9a108a3a95)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:37.382348 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600119f-9fc7-491a-a09f-b6376ab63dd6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:37.386314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 575/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:37 2022] GET /identity/v3/users/31e63320afd4449aacdbbf9a108a3a95 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:37.389765 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b068b32-fdf8-481c-b7b0-cba3a36760af None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:37.486966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b068b32-fdf8-481c-b7b0-cba3a36760af None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:37.511433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:37.520390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c68db-baf9-4c19-8b42-18131b539f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:37.520985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c68db-baf9-4c19-8b42-18131b539f38 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:37.521327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c68db-baf9-4c19-8b42-18131b539f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:37.521600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c68db-baf9-4c19-8b42-18131b539f38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:37.585708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c68db-baf9-4c19-8b42-18131b539f38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:37.587131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c68db-baf9-4c19-8b42-18131b539f38 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:37.679452 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 584/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:42:37.685324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600119f-9fc7-491a-a09f-b6376ab63dd6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:37.720473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:37.782906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600119f-9fc7-491a-a09f-b6376ab63dd6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:37.815769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c62d2-49b8-4fb4-913a-581d49c15aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:37.816390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c62d2-49b8-4fb4-913a-581d49c15aad None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:37.816668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c62d2-49b8-4fb4-913a-581d49c15aad None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:37.816936 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c62d2-49b8-4fb4-913a-581d49c15aad None admin] PATH_INFO: `/v3/projects/f61d62eeb7664708bf1d53e845c69ffb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:37.818322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c62d2-49b8-4fb4-913a-581d49c15aad None admin] RBAC: Authorizing `identity:get_project(project_id=f61d62eeb7664708bf1d53e845c69ffb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:37.819710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c62d2-49b8-4fb4-913a-581d49c15aad None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:37.821332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 576/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:37 2022] GET /identity/v3/projects/f61d62eeb7664708bf1d53e845c69ffb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:37.827284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c68db-baf9-4c19-8b42-18131b539f38 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:37.852612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:37.920810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c68db-baf9-4c19-8b42-18131b539f38 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:37.954092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809125b0-43bd-4a9d-b291-3c2db8fd9066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:37.954613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809125b0-43bd-4a9d-b291-3c2db8fd9066 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:37.954888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809125b0-43bd-4a9d-b291-3c2db8fd9066 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:37.955158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809125b0-43bd-4a9d-b291-3c2db8fd9066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:37.955879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809125b0-43bd-4a9d-b291-3c2db8fd9066 None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=f61d62eeb7664708bf1d53e845c69ffb, user.id=31e63320afd4449aacdbbf9a108a3a95) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:37.956181 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809125b0-43bd-4a9d-b291-3c2db8fd9066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:37.957616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809125b0-43bd-4a9d-b291-3c2db8fd9066 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:37.963092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 585/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:42:37 2022] GET /identity/v3/role_assignments?user.id=31e63320afd4449aacdbbf9a108a3a95&scope.project.id=f61d62eeb7664708bf1d53e845c69ffb&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:39.964814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888d7a0-f964-4afb-885b-a62fcc5f94a0 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:39.965408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888d7a0-f964-4afb-885b-a62fcc5f94a0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:39.965408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888d7a0-f964-4afb-885b-a62fcc5f94a0 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:39.966724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 577/1162] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:39.972061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396eb0c-05cb-43cf-8df2-46ffa3882329 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:39.972497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396eb0c-05cb-43cf-8df2-46ffa3882329 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:39.972613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396eb0c-05cb-43cf-8df2-46ffa3882329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:39.990190 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b396eb0c-05cb-43cf-8df2-46ffa3882329 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:39.991389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b396eb0c-05cb-43cf-8df2-46ffa3882329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:40.056309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 586/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:40.082858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888d7a0-f964-4afb-885b-a62fcc5f94a0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:40.182414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888d7a0-f964-4afb-885b-a62fcc5f94a0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:40.184763 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:40.215896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caaba9-eb7c-4ad0-ab5e-d6edd15db97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:40.216431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caaba9-eb7c-4ad0-ab5e-d6edd15db97a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:40.216708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caaba9-eb7c-4ad0-ab5e-d6edd15db97a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:40.216979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caaba9-eb7c-4ad0-ab5e-d6edd15db97a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:40.282478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caaba9-eb7c-4ad0-ab5e-d6edd15db97a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:40.283890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09caaba9-eb7c-4ad0-ab5e-d6edd15db97a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:40.378461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 578/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:42:40.384383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396eb0c-05cb-43cf-8df2-46ffa3882329 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:40.405035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:40.480410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396eb0c-05cb-43cf-8df2-46ffa3882329 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:40.512769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e2213-36b5-4da7-adcd-47ee0999b8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:40.513408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e2213-36b5-4da7-adcd-47ee0999b8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:40.513692 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e2213-36b5-4da7-adcd-47ee0999b8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:40.513965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e2213-36b5-4da7-adcd-47ee0999b8e3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:40.520505 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e2213-36b5-4da7-adcd-47ee0999b8e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:40.521971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e2213-36b5-4da7-adcd-47ee0999b8e3 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-bc3e2213-36b5-4da7-adcd-47ee0999b8e3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:40.534647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:40.535908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:40.537081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 587/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Aug 11 09:42:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:40.541602 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caaba9-eb7c-4ad0-ab5e-d6edd15db97a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:40.544606 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:40.638472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caaba9-eb7c-4ad0-ab5e-d6edd15db97a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:40.670531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b08ca-22ea-401e-9052-13fb2273cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:40.671135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b08ca-22ea-401e-9052-13fb2273cac8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:40.671414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b08ca-22ea-401e-9052-13fb2273cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:40.671685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b08ca-22ea-401e-9052-13fb2273cac8 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:40.672952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b08ca-22ea-401e-9052-13fb2273cac8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:40.673267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b08ca-22ea-401e-9052-13fb2273cac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:40.674599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b08ca-22ea-401e-9052-13fb2273cac8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:40.681569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 579/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Aug 11 09:42:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:42.652983 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf743-0f49-4f09-a37d-14d795a70442 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:42.653514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf743-0f49-4f09-a37d-14d795a70442 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:42.653608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebf743-0f49-4f09-a37d-14d795a70442 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:42.654862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 588/1167] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:42.660573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f740f35-2178-48f0-886f-601707e307d3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:42.660987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f740f35-2178-48f0-886f-601707e307d3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:42.661141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f740f35-2178-48f0-886f-601707e307d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:42.678984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3f740f35-2178-48f0-886f-601707e307d3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:42.680154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3f740f35-2178-48f0-886f-601707e307d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:42.740543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:42.745705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 580/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:42.769112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebf743-0f49-4f09-a37d-14d795a70442 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:42.847198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:42.865288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebf743-0f49-4f09-a37d-14d795a70442 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:42.897610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f0d38-ff92-4e35-a4a1-1cd14a010d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:42.898198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f0d38-ff92-4e35-a4a1-1cd14a010d28 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:42.898475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f0d38-ff92-4e35-a4a1-1cd14a010d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:42.898745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f0d38-ff92-4e35-a4a1-1cd14a010d28 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:42.899394 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f0d38-ff92-4e35-a4a1-1cd14a010d28 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:42.900729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f0d38-ff92-4e35-a4a1-1cd14a010d28 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:42.970556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 589/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Thu Aug 11 09:42:42 2022] POST /identity/v3/domains => generated 267 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:42:44.963643 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362954d-7f15-4745-b6f2-d11a19a28ead None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:44.964189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362954d-7f15-4745-b6f2-d11a19a28ead None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:44.964189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362954d-7f15-4745-b6f2-d11a19a28ead None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:44.965580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 581/1170] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:44.971214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877a2c3-fddc-4f68-8c7d-10e98770e30d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:44.971627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877a2c3-fddc-4f68-8c7d-10e98770e30d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:44.971762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877a2c3-fddc-4f68-8c7d-10e98770e30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:44.989704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d877a2c3-fddc-4f68-8c7d-10e98770e30d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:44.990883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d877a2c3-fddc-4f68-8c7d-10e98770e30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:45.053688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 590/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:45.076151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362954d-7f15-4745-b6f2-d11a19a28ead None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:45.174745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362954d-7f15-4745-b6f2-d11a19a28ead None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:45.207242 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571ffd6-1865-4673-b323-1a2b7accdb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:45.207864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571ffd6-1865-4673-b323-1a2b7accdb67 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:45.208142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571ffd6-1865-4673-b323-1a2b7accdb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:45.208411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571ffd6-1865-4673-b323-1a2b7accdb67 None admin] PATH_INFO: `/v3/domains/b04b56e24f1d4462ac34f04d53bf9c5a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:45.218351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571ffd6-1865-4673-b323-1a2b7accdb67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b04b56e24f1d4462ac34f04d53bf9c5a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:45.220372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571ffd6-1865-4673-b323-1a2b7accdb67 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:45.221994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 582/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Aug 11 09:42:45 2022] GET /identity/v3/domains/b04b56e24f1d4462ac34f04d53bf9c5a => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:45.227783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877a2c3-fddc-4f68-8c7d-10e98770e30d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:45.293290 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:45.322340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877a2c3-fddc-4f68-8c7d-10e98770e30d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:45.356212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea87c95-822d-405d-9cb9-ab5e6642a01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:45.356805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea87c95-822d-405d-9cb9-ab5e6642a01a None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:45.357116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea87c95-822d-405d-9cb9-ab5e6642a01a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:45.357397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea87c95-822d-405d-9cb9-ab5e6642a01a None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:45.358046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea87c95-822d-405d-9cb9-ab5e6642a01a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:45.359487 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea87c95-822d-405d-9cb9-ab5e6642a01a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:45.427458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 591/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:42:45 2022] POST /identity/v3/projects => generated 361 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:42:47.429710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846e3ed-3930-4959-8159-64f3a5023ed7 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:47.430253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846e3ed-3930-4959-8159-64f3a5023ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:47.430253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846e3ed-3930-4959-8159-64f3a5023ed7 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:47.431587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 583/1174] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:47.436687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956763c2-3bc6-46e2-a8f6-d9c21cccdb15 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:47.437128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956763c2-3bc6-46e2-a8f6-d9c21cccdb15 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:47.437761 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956763c2-3bc6-46e2-a8f6-d9c21cccdb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:47.456225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-956763c2-3bc6-46e2-a8f6-d9c21cccdb15 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:47.457451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-956763c2-3bc6-46e2-a8f6-d9c21cccdb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:47.520723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 592/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:47.544712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846e3ed-3930-4959-8159-64f3a5023ed7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:47.603143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:47.645846 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846e3ed-3930-4959-8159-64f3a5023ed7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:47.678306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272e90f-5f89-4272-9b47-bd3fad2dace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:47.678923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272e90f-5f89-4272-9b47-bd3fad2dace0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:47.679203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272e90f-5f89-4272-9b47-bd3fad2dace0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:47.679477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272e90f-5f89-4272-9b47-bd3fad2dace0 None admin] PATH_INFO: `/v3/domains/b04b56e24f1d4462ac34f04d53bf9c5a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:47.680591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272e90f-5f89-4272-9b47-bd3fad2dace0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b04b56e24f1d4462ac34f04d53bf9c5a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:47.682024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272e90f-5f89-4272-9b47-bd3fad2dace0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:47.683678 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 584/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Aug 11 09:42:47 2022] GET /identity/v3/domains/b04b56e24f1d4462ac34f04d53bf9c5a => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:47.690158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956763c2-3bc6-46e2-a8f6-d9c21cccdb15 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:47.740112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:47.786054 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956763c2-3bc6-46e2-a8f6-d9c21cccdb15 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:47.819606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11162d64-3ba8-41f4-b76a-272715ec373d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:47.820200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11162d64-3ba8-41f4-b76a-272715ec373d None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:47.820478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11162d64-3ba8-41f4-b76a-272715ec373d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:47.820748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11162d64-3ba8-41f4-b76a-272715ec373d None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:47.821411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11162d64-3ba8-41f4-b76a-272715ec373d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:47.822837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11162d64-3ba8-41f4-b76a-272715ec373d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:47.900259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 593/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:42:47 2022] POST /identity/v3/users => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:42:49.934721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c0f20-4d99-4b74-ba15-5afaad748ef9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:49.935278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c0f20-4d99-4b74-ba15-5afaad748ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:49.935278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c0f20-4d99-4b74-ba15-5afaad748ef9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:49.936604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 585/1178] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:42:49.942755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148179b-f4e2-4461-a39a-e58a5d8e021f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:49.943136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148179b-f4e2-4461-a39a-e58a5d8e021f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:49.943301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148179b-f4e2-4461-a39a-e58a5d8e021f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:49.961821 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6148179b-f4e2-4461-a39a-e58a5d8e021f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:49.963013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6148179b-f4e2-4461-a39a-e58a5d8e021f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:50.027383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 594/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:50.050749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c0f20-4d99-4b74-ba15-5afaad748ef9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:50.150072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c0f20-4d99-4b74-ba15-5afaad748ef9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:50.175018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:50.183416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4a1eb-1705-4dd3-8f14-62657988cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:50.184101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4a1eb-1705-4dd3-8f14-62657988cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:50.184444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4a1eb-1705-4dd3-8f14-62657988cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:50.184772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4a1eb-1705-4dd3-8f14-62657988cfe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:50.196008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4a1eb-1705-4dd3-8f14-62657988cfe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-52c4a1eb-1705-4dd3-8f14-62657988cfe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:50.201185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:50.202287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:50.203714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 586/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:42:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:50.209150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148179b-f4e2-4461-a39a-e58a5d8e021f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:50.246395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:50.302687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148179b-f4e2-4461-a39a-e58a5d8e021f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:50.336099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c26b1-d74e-42e2-96fb-20ac5d08840d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:50.336694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c26b1-d74e-42e2-96fb-20ac5d08840d None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:50.336973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c26b1-d74e-42e2-96fb-20ac5d08840d None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:50.337280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c26b1-d74e-42e2-96fb-20ac5d08840d None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:50.337936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c26b1-d74e-42e2-96fb-20ac5d08840d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:50.338236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c26b1-d74e-42e2-96fb-20ac5d08840d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:50.339585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c26b1-d74e-42e2-96fb-20ac5d08840d None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:50.348039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 595/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:42:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:50.353872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4a1eb-1705-4dd3-8f14-62657988cfe7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:50.449074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4a1eb-1705-4dd3-8f14-62657988cfe7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:50.481444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1729e5c-93bd-4b74-a91a-0f3060046aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:50.482052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1729e5c-93bd-4b74-a91a-0f3060046aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:50.482330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1729e5c-93bd-4b74-a91a-0f3060046aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:50.482600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1729e5c-93bd-4b74-a91a-0f3060046aa7 None admin] PATH_INFO: `/v3/users/a31f80cbada049a1a9f53ad68490ab85` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:50.496412 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1729e5c-93bd-4b74-a91a-0f3060046aa7 None admin] RBAC: Authorizing `identity:get_user(user_id=a31f80cbada049a1a9f53ad68490ab85)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:50.497856 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1729e5c-93bd-4b74-a91a-0f3060046aa7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:50.499466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 587/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:50 2022] GET /identity/v3/users/a31f80cbada049a1a9f53ad68490ab85 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:50.505929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c26b1-d74e-42e2-96fb-20ac5d08840d None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:50.599022 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:50.601854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c26b1-d74e-42e2-96fb-20ac5d08840d None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:50.634449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ddbc6-97d4-4f17-8ded-654c0f127022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:50.635042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ddbc6-97d4-4f17-8ded-654c0f127022 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:50.635316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ddbc6-97d4-4f17-8ded-654c0f127022 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:50.635585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ddbc6-97d4-4f17-8ded-654c0f127022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:50.700068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ddbc6-97d4-4f17-8ded-654c0f127022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:50.701469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ddbc6-97d4-4f17-8ded-654c0f127022 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:50.790927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:50.794400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 596/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:42:50.800803 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1729e5c-93bd-4b74-a91a-0f3060046aa7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:50.845686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:50.897682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1729e5c-93bd-4b74-a91a-0f3060046aa7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:50.930050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6841025-093c-4624-889e-71999742bacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:50.930667 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6841025-093c-4624-889e-71999742bacc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:50.930945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6841025-093c-4624-889e-71999742bacc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:50.931216 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6841025-093c-4624-889e-71999742bacc None admin] PATH_INFO: `/v3/projects/d744fbb834f943caa9601e7036ec0297` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:50.932315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6841025-093c-4624-889e-71999742bacc None admin] RBAC: Authorizing `identity:get_project(project_id=d744fbb834f943caa9601e7036ec0297)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:50.933750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6841025-093c-4624-889e-71999742bacc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:50.935339 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 588/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:50 2022] GET /identity/v3/projects/d744fbb834f943caa9601e7036ec0297 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:50.941863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ddbc6-97d4-4f17-8ded-654c0f127022 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:51.036580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ddbc6-97d4-4f17-8ded-654c0f127022 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:51.069094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53650466-d9a0-4909-b731-e6e7ed6c0a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:51.069680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650466-d9a0-4909-b731-e6e7ed6c0a3f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:51.069956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650466-d9a0-4909-b731-e6e7ed6c0a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:51.070226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53650466-d9a0-4909-b731-e6e7ed6c0a3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:51.070943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53650466-d9a0-4909-b731-e6e7ed6c0a3f None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=d744fbb834f943caa9601e7036ec0297, user.id=a31f80cbada049a1a9f53ad68490ab85) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:51.071244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53650466-d9a0-4909-b731-e6e7ed6c0a3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:51.072647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53650466-d9a0-4909-b731-e6e7ed6c0a3f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:51.077830 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 597/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:42:50 2022] GET /identity/v3/role_assignments?user.id=a31f80cbada049a1a9f53ad68490ab85&scope.project.id=d744fbb834f943caa9601e7036ec0297&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:53.035341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487bb90-5043-44b6-af61-02daa8592fab None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:53.035894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487bb90-5043-44b6-af61-02daa8592fab None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:53.035894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487bb90-5043-44b6-af61-02daa8592fab None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:53.037298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 589/1186] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:53.042449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84697da6-618c-428e-a75b-b3f388592c83 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:53.042861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84697da6-618c-428e-a75b-b3f388592c83 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:53.042986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84697da6-618c-428e-a75b-b3f388592c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:53.061300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-84697da6-618c-428e-a75b-b3f388592c83 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:53.063056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-84697da6-618c-428e-a75b-b3f388592c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:53.120185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:53.128285 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 598/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:53.151059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487bb90-5043-44b6-af61-02daa8592fab None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:53.252226 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487bb90-5043-44b6-af61-02daa8592fab None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:53.266095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:53.287700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb90eb-6283-4f6c-9f05-e3c2c1d49ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:53.288267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb90eb-6283-4f6c-9f05-e3c2c1d49ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:53.288548 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb90eb-6283-4f6c-9f05-e3c2c1d49ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:53.288819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb90eb-6283-4f6c-9f05-e3c2c1d49ad5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:53.299102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb90eb-6283-4f6c-9f05-e3c2c1d49ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-44eb90eb-6283-4f6c-9f05-e3c2c1d49ad5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:53.303802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:53.304865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:42:53.305996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 590/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:42:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:53.311111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84697da6-618c-428e-a75b-b3f388592c83 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:53.405443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84697da6-618c-428e-a75b-b3f388592c83 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:53.437501 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b00b0e-a57a-482c-b687-9b72aff009f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:53.438211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b00b0e-a57a-482c-b687-9b72aff009f8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:53.438486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b00b0e-a57a-482c-b687-9b72aff009f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:53.438752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b00b0e-a57a-482c-b687-9b72aff009f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:53.439400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b00b0e-a57a-482c-b687-9b72aff009f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:53.439694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b00b0e-a57a-482c-b687-9b72aff009f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:53.441052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b00b0e-a57a-482c-b687-9b72aff009f8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:53.448916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 599/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:42:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:53.454841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb90eb-6283-4f6c-9f05-e3c2c1d49ad5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:53.552936 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb90eb-6283-4f6c-9f05-e3c2c1d49ad5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:53.585834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d173b-e4e4-49e2-9a12-68bdae06728d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:53.586445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d173b-e4e4-49e2-9a12-68bdae06728d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:53.586722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d173b-e4e4-49e2-9a12-68bdae06728d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:53.586993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d173b-e4e4-49e2-9a12-68bdae06728d None admin] PATH_INFO: `/v3/users/a31f80cbada049a1a9f53ad68490ab85` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:53.588085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d173b-e4e4-49e2-9a12-68bdae06728d None admin] RBAC: Authorizing `identity:get_user(user_id=a31f80cbada049a1a9f53ad68490ab85)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:53.589517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d173b-e4e4-49e2-9a12-68bdae06728d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:53.591051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 591/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:53 2022] GET /identity/v3/users/a31f80cbada049a1a9f53ad68490ab85 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:53.597406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b00b0e-a57a-482c-b687-9b72aff009f8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:53.651283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:53.694569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b00b0e-a57a-482c-b687-9b72aff009f8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:53.727447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cc109-3fb6-4e61-8a0b-f234a2ae83ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:53.728098 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cc109-3fb6-4e61-8a0b-f234a2ae83ce None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:53.728379 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cc109-3fb6-4e61-8a0b-f234a2ae83ce None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:53.728651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cc109-3fb6-4e61-8a0b-f234a2ae83ce None admin] PATH_INFO: `/v3/projects/d744fbb834f943caa9601e7036ec0297` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:53.729790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cc109-3fb6-4e61-8a0b-f234a2ae83ce None admin] RBAC: Authorizing `identity:get_project(project_id=d744fbb834f943caa9601e7036ec0297)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:53.731201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cc109-3fb6-4e61-8a0b-f234a2ae83ce None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:53.732795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 600/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:53 2022] GET /identity/v3/projects/d744fbb834f943caa9601e7036ec0297 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:53.738733 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d173b-e4e4-49e2-9a12-68bdae06728d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:53.779154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:53.836130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d173b-e4e4-49e2-9a12-68bdae06728d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:53.868760 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49f3be-7438-4bd8-a1bf-f3c9df76a6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:53.869419 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49f3be-7438-4bd8-a1bf-f3c9df76a6ad None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:53.869702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49f3be-7438-4bd8-a1bf-f3c9df76a6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:53.869979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49f3be-7438-4bd8-a1bf-f3c9df76a6ad None admin] PATH_INFO: `/v3/projects/d744fbb834f943caa9601e7036ec0297/users/a31f80cbada049a1a9f53ad68490ab85/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:53.871856 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49f3be-7438-4bd8-a1bf-f3c9df76a6ad None admin] RBAC: Authorizing `identity:create_grant(project_id=d744fbb834f943caa9601e7036ec0297, user_id=a31f80cbada049a1a9f53ad68490ab85, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:53.873279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49f3be-7438-4bd8-a1bf-f3c9df76a6ad None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:53.922991 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 592/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:42:53 2022] PUT /identity/v3/projects/d744fbb834f943caa9601e7036ec0297/users/a31f80cbada049a1a9f53ad68490ab85/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:42:55.895051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd587ca-cdb6-4c27-93a1-1b91652fd096 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:55.895643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd587ca-cdb6-4c27-93a1-1b91652fd096 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:55.895643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd587ca-cdb6-4c27-93a1-1b91652fd096 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:55.896924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 601/1193] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:42:55.901954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720dffa-30bb-4e84-ab61-421ca84e7b9a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:55.902338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720dffa-30bb-4e84-ab61-421ca84e7b9a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:55.902498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720dffa-30bb-4e84-ab61-421ca84e7b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:55.920888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c720dffa-30bb-4e84-ab61-421ca84e7b9a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:42:55.922165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c720dffa-30bb-4e84-ab61-421ca84e7b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:42:55.986501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 593/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:42:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:42:56.009163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd587ca-cdb6-4c27-93a1-1b91652fd096 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:56.107218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd587ca-cdb6-4c27-93a1-1b91652fd096 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:56.129161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:56.139178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50372c-4a4d-4d29-a904-b7fe703b7116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:56.143833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50372c-4a4d-4d29-a904-b7fe703b7116 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:56.143833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50372c-4a4d-4d29-a904-b7fe703b7116 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:56.143833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50372c-4a4d-4d29-a904-b7fe703b7116 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:56.151141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50372c-4a4d-4d29-a904-b7fe703b7116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-0f50372c-4a4d-4d29-a904-b7fe703b7116 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:42:56.156530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:42:56.157915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:42:56.158957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 602/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:42:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:42:56.163588 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720dffa-30bb-4e84-ab61-421ca84e7b9a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:56.196001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:56.267523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720dffa-30bb-4e84-ab61-421ca84e7b9a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:56.302283 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b242370-aabb-4c59-b714-7eacd7a127ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:56.302901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b242370-aabb-4c59-b714-7eacd7a127ed None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:56.303187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b242370-aabb-4c59-b714-7eacd7a127ed None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:56.303464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b242370-aabb-4c59-b714-7eacd7a127ed None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:56.304128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b242370-aabb-4c59-b714-7eacd7a127ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:56.304435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b242370-aabb-4c59-b714-7eacd7a127ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:56.305861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b242370-aabb-4c59-b714-7eacd7a127ed None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:56.314592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 594/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:42:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:56.320631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50372c-4a4d-4d29-a904-b7fe703b7116 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:56.416683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50372c-4a4d-4d29-a904-b7fe703b7116 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:56.449843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae94af2-35bf-47d7-a4f7-24e916ec252a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:56.450459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae94af2-35bf-47d7-a4f7-24e916ec252a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:56.450741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae94af2-35bf-47d7-a4f7-24e916ec252a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:56.451016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae94af2-35bf-47d7-a4f7-24e916ec252a None admin] PATH_INFO: `/v3/users/a31f80cbada049a1a9f53ad68490ab85` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:56.452127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae94af2-35bf-47d7-a4f7-24e916ec252a None admin] RBAC: Authorizing `identity:get_user(user_id=a31f80cbada049a1a9f53ad68490ab85)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:56.454327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae94af2-35bf-47d7-a4f7-24e916ec252a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:56.455884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 603/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Aug 11 09:42:56 2022] GET /identity/v3/users/a31f80cbada049a1a9f53ad68490ab85 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:56.461584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b242370-aabb-4c59-b714-7eacd7a127ed None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:56.548638 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:56.556756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b242370-aabb-4c59-b714-7eacd7a127ed None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:56.589542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be454c0e-fd82-40b1-9b96-30a99242e3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:56.590139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be454c0e-fd82-40b1-9b96-30a99242e3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:56.590417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be454c0e-fd82-40b1-9b96-30a99242e3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:56.590686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be454c0e-fd82-40b1-9b96-30a99242e3e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:56.654744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be454c0e-fd82-40b1-9b96-30a99242e3e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:56.656483 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be454c0e-fd82-40b1-9b96-30a99242e3e2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:56.741717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:56.750468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 595/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:42:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:42:56.756379 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae94af2-35bf-47d7-a4f7-24e916ec252a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:56.808576 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:42:56.855849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae94af2-35bf-47d7-a4f7-24e916ec252a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:56.888036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e6d90-3779-45a7-9f9f-daa58a919d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:56.888657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e6d90-3779-45a7-9f9f-daa58a919d48 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:56.888936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e6d90-3779-45a7-9f9f-daa58a919d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:56.889239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e6d90-3779-45a7-9f9f-daa58a919d48 None admin] PATH_INFO: `/v3/projects/d744fbb834f943caa9601e7036ec0297` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:56.891119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e6d90-3779-45a7-9f9f-daa58a919d48 None admin] RBAC: Authorizing `identity:get_project(project_id=d744fbb834f943caa9601e7036ec0297)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:56.892537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e6d90-3779-45a7-9f9f-daa58a919d48 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:56.894164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 604/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Aug 11 09:42:56 2022] GET /identity/v3/projects/d744fbb834f943caa9601e7036ec0297 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:42:56.900145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be454c0e-fd82-40b1-9b96-30a99242e3e2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:42:56.996522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be454c0e-fd82-40b1-9b96-30a99242e3e2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:42:57.028736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa4676-4e56-4b07-95cb-8138029f3e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:42:57.029355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4676-4e56-4b07-95cb-8138029f3e54 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:42:57.029635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4676-4e56-4b07-95cb-8138029f3e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:42:57.029906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa4676-4e56-4b07-95cb-8138029f3e54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:42:57.031142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa4676-4e56-4b07-95cb-8138029f3e54 None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=d744fbb834f943caa9601e7036ec0297, user.id=a31f80cbada049a1a9f53ad68490ab85) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:42:57.031449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa4676-4e56-4b07-95cb-8138029f3e54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:42:57.032842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa4676-4e56-4b07-95cb-8138029f3e54 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:42:57.038388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 596/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:42:56 2022] GET /identity/v3/role_assignments?user.id=a31f80cbada049a1a9f53ad68490ab85&scope.project.id=d744fbb834f943caa9601e7036ec0297&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:43:06.883960 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=40 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:43:07.036067 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=72 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:08.760815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5eb5-f317-41e9-ac7a-98e75d7ea20b None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:08.761467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5eb5-f317-41e9-ac7a-98e75d7ea20b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:08.761467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead5eb5-f317-41e9-ac7a-98e75d7ea20b None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:08.762750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 605/1201] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:44:08.768054 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61c379-8573-4287-bd62-19fde3bada42 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:08.768493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61c379-8573-4287-bd62-19fde3bada42 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:08.768606 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61c379-8573-4287-bd62-19fde3bada42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:08.786230 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5d61c379-8573-4287-bd62-19fde3bada42 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:44:08.787406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5d61c379-8573-4287-bd62-19fde3bada42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:44:08.852287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 597/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:44:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:44:08.876826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead5eb5-f317-41e9-ac7a-98e75d7ea20b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:08.977608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead5eb5-f317-41e9-ac7a-98e75d7ea20b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:09.010463 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689eecf-99c4-49a2-9408-b55089ef8ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:09.012943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689eecf-99c4-49a2-9408-b55089ef8ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:09.012943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689eecf-99c4-49a2-9408-b55089ef8ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:09.012943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689eecf-99c4-49a2-9408-b55089ef8ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:09.021724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689eecf-99c4-49a2-9408-b55089ef8ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:09.023144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689eecf-99c4-49a2-9408-b55089ef8ca8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-8689eecf-99c4-49a2-9408-b55089ef8ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:09.036562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:09.037769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:09.038917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:09.038917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:09.038917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:09.038917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 606/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:09.043816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61c379-8573-4287-bd62-19fde3bada42 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:09.147932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61c379-8573-4287-bd62-19fde3bada42 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:09.176517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:09.182188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc57f3b-ae26-4e3e-b105-8bd800bad131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:09.182796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc57f3b-ae26-4e3e-b105-8bd800bad131 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:09.183081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc57f3b-ae26-4e3e-b105-8bd800bad131 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:09.183357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc57f3b-ae26-4e3e-b105-8bd800bad131 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:09.184038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc57f3b-ae26-4e3e-b105-8bd800bad131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:09.184360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc57f3b-ae26-4e3e-b105-8bd800bad131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:09.185704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc57f3b-ae26-4e3e-b105-8bd800bad131 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:09.197531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 598/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:44:09.203335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689eecf-99c4-49a2-9408-b55089ef8ca8 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:09.255793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:09.301158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689eecf-99c4-49a2-9408-b55089ef8ca8 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:09.333028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cf997-aaa6-4792-a0c0-d346f4cbbea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:09.333633 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf997-aaa6-4792-a0c0-d346f4cbbea7 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:09.333910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf997-aaa6-4792-a0c0-d346f4cbbea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:09.334180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf997-aaa6-4792-a0c0-d346f4cbbea7 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:09.334806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cf997-aaa6-4792-a0c0-d346f4cbbea7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:09.336298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cf997-aaa6-4792-a0c0-d346f4cbbea7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:09.648602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 607/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Aug 11 09:44:09 2022] POST /identity/v3/users => generated 251 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:44:11.669578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f431a-1776-4127-958f-f260a0b5f9c0 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:11.670149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f431a-1776-4127-958f-f260a0b5f9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:11.670149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f431a-1776-4127-958f-f260a0b5f9c0 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:11.671499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 599/1206] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:44:11.676920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562066c2-c206-4615-9864-b8581e2ccb4f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:11.677373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562066c2-c206-4615-9864-b8581e2ccb4f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:11.677473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562066c2-c206-4615-9864-b8581e2ccb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:11.696122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-562066c2-c206-4615-9864-b8581e2ccb4f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:44:11.697348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-562066c2-c206-4615-9864-b8581e2ccb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:44:11.765837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 608/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:44:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:44:11.788851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f431a-1776-4127-958f-f260a0b5f9c0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:11.889183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f431a-1776-4127-958f-f260a0b5f9c0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:11.923097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611e1e5-4917-46bc-aae4-d0bffabc043a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:11.923654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611e1e5-4917-46bc-aae4-d0bffabc043a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:11.923929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611e1e5-4917-46bc-aae4-d0bffabc043a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:11.924198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611e1e5-4917-46bc-aae4-d0bffabc043a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:11.935156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611e1e5-4917-46bc-aae4-d0bffabc043a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a611e1e5-4917-46bc-aae4-d0bffabc043a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:44:11.939798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:11.940902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:11.942522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 600/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:44:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:11.947974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562066c2-c206-4615-9864-b8581e2ccb4f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:11.987967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:12.046715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562066c2-c206-4615-9864-b8581e2ccb4f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:12.079275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d52d9-559f-4bc1-b61c-ffcb6aa8126f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:12.079862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d52d9-559f-4bc1-b61c-ffcb6aa8126f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:12.080138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d52d9-559f-4bc1-b61c-ffcb6aa8126f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:12.080406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d52d9-559f-4bc1-b61c-ffcb6aa8126f None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:12.081195 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d52d9-559f-4bc1-b61c-ffcb6aa8126f None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:12.081493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d52d9-559f-4bc1-b61c-ffcb6aa8126f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:12.083355 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d52d9-559f-4bc1-b61c-ffcb6aa8126f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:12.091519 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 609/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:44:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:12.097981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611e1e5-4917-46bc-aae4-d0bffabc043a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:12.149346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:12.198654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611e1e5-4917-46bc-aae4-d0bffabc043a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:12.233301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e98ea0-2df9-4c6f-bb75-07f9a3f1d3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:12.233926 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e98ea0-2df9-4c6f-bb75-07f9a3f1d3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:12.234210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e98ea0-2df9-4c6f-bb75-07f9a3f1d3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:12.234511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e98ea0-2df9-4c6f-bb75-07f9a3f1d3f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:12.244702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e98ea0-2df9-4c6f-bb75-07f9a3f1d3f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:12.246137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e98ea0-2df9-4c6f-bb75-07f9a3f1d3f5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f7e98ea0-2df9-4c6f-bb75-07f9a3f1d3f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:12.260291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:12.261428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:12.262584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:12.262584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:12.262584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:12.262584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 601/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:12.267129 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d52d9-559f-4bc1-b61c-ffcb6aa8126f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:12.364007 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:12.366943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d52d9-559f-4bc1-b61c-ffcb6aa8126f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:12.400820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f590f18-b17d-47fd-b19e-7b8218d8e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:12.401691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590f18-b17d-47fd-b19e-7b8218d8e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:12.402197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590f18-b17d-47fd-b19e-7b8218d8e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:12.402788 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f590f18-b17d-47fd-b19e-7b8218d8e5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:12.403497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f590f18-b17d-47fd-b19e-7b8218d8e5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:12.403801 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f590f18-b17d-47fd-b19e-7b8218d8e5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:12.405251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f590f18-b17d-47fd-b19e-7b8218d8e5e4 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:12.417247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 610/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:12.422990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e98ea0-2df9-4c6f-bb75-07f9a3f1d3f5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:12.524360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e98ea0-2df9-4c6f-bb75-07f9a3f1d3f5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:12.525913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:12.557486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa177f8-32aa-49e7-b556-6f606132a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:12.558104 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa177f8-32aa-49e7-b556-6f606132a359 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:12.558380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa177f8-32aa-49e7-b556-6f606132a359 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:12.558650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa177f8-32aa-49e7-b556-6f606132a359 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:12.565273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa177f8-32aa-49e7-b556-6f606132a359 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:12.566736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa177f8-32aa-49e7-b556-6f606132a359 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-9aa177f8-32aa-49e7-b556-6f606132a359 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:44:12.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:12.583759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:12.584945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:12.584945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:12.584945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:44:12.584945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:44:12.584945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:44:12.584945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:44:12.584945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:44:12.584945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:44:12.584945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:12.584945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:12.584945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 602/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:12.585709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f590f18-b17d-47fd-b19e-7b8218d8e5e4 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:12.683775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f590f18-b17d-47fd-b19e-7b8218d8e5e4 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:12.717294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c723c-1eb5-45b7-895c-8a406b521be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:12.717894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c723c-1eb5-45b7-895c-8a406b521be5 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:12.718170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c723c-1eb5-45b7-895c-8a406b521be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:12.718441 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c723c-1eb5-45b7-895c-8a406b521be5 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:12.719163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c723c-1eb5-45b7-895c-8a406b521be5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:12.719465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c723c-1eb5-45b7-895c-8a406b521be5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:12.720887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c723c-1eb5-45b7-895c-8a406b521be5 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:12.737699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 611/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Aug 11 09:44:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:12.743627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa177f8-32aa-49e7-b556-6f606132a359 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:12.840372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa177f8-32aa-49e7-b556-6f606132a359 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:12.873217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6f84d-7aca-41b2-8af5-54d6fdd7da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:12.873822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6f84d-7aca-41b2-8af5-54d6fdd7da5d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:12.874101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6f84d-7aca-41b2-8af5-54d6fdd7da5d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:12.874374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6f84d-7aca-41b2-8af5-54d6fdd7da5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:12.903884 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:12.940892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6f84d-7aca-41b2-8af5-54d6fdd7da5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:12.943234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6f84d-7aca-41b2-8af5-54d6fdd7da5d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:13.042832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 603/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:44:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:44:13.048600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c723c-1eb5-45b7-895c-8a406b521be5 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:13.068029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:13.145576 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c723c-1eb5-45b7-895c-8a406b521be5 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:13.178649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befef0e-3396-4959-92da-bd4bdc94ac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:13.179292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befef0e-3396-4959-92da-bd4bdc94ac59 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:13.179569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befef0e-3396-4959-92da-bd4bdc94ac59 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:13.179847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befef0e-3396-4959-92da-bd4bdc94ac59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:13.189965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befef0e-3396-4959-92da-bd4bdc94ac59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:13.191983 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befef0e-3396-4959-92da-bd4bdc94ac59 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-2befef0e-3396-4959-92da-bd4bdc94ac59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:13.207405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:13.208654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:13.210142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:13.210142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:13.210142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:13.210142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 612/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:13.214324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6f84d-7aca-41b2-8af5-54d6fdd7da5d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:13.275112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:13.313377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6f84d-7aca-41b2-8af5-54d6fdd7da5d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:13.346178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caeea7d-7275-424c-8d9d-e6f70726946f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:13.346771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeea7d-7275-424c-8d9d-e6f70726946f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:13.347051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeea7d-7275-424c-8d9d-e6f70726946f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:13.347351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeea7d-7275-424c-8d9d-e6f70726946f None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:13.347993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caeea7d-7275-424c-8d9d-e6f70726946f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:13.348317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caeea7d-7275-424c-8d9d-e6f70726946f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:13.349686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caeea7d-7275-424c-8d9d-e6f70726946f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:13.361807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 604/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:13.367878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befef0e-3396-4959-92da-bd4bdc94ac59 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:13.435595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:13.465717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befef0e-3396-4959-92da-bd4bdc94ac59 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:13.498425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74f162-9be8-4b99-9552-0fcf278328ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:13.499045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74f162-9be8-4b99-9552-0fcf278328ff None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:13.499325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74f162-9be8-4b99-9552-0fcf278328ff None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:13.499597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74f162-9be8-4b99-9552-0fcf278328ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:13.506628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74f162-9be8-4b99-9552-0fcf278328ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:13.508061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab74f162-9be8-4b99-9552-0fcf278328ff None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-ab74f162-9be8-4b99-9552-0fcf278328ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:44:13.518891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:13.519969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:13.521138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:44:13.521138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:44:13.521138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:44:13.521138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:44:13.521138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:44:13.521138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:44:13.521138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:13.521138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:13.521138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 613/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:44:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:13.526058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caeea7d-7275-424c-8d9d-e6f70726946f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:13.624691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caeea7d-7275-424c-8d9d-e6f70726946f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:13.646614 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:13.657655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fb749-bb6a-46ab-91d3-1f104a43c349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:13.658263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fb749-bb6a-46ab-91d3-1f104a43c349 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:13.658578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fb749-bb6a-46ab-91d3-1f104a43c349 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:13.658851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fb749-bb6a-46ab-91d3-1f104a43c349 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:13.659535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fb749-bb6a-46ab-91d3-1f104a43c349 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:13.660399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fb749-bb6a-46ab-91d3-1f104a43c349 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:13.661835 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fb749-bb6a-46ab-91d3-1f104a43c349 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:13.673128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 605/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:44:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:13.679950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74f162-9be8-4b99-9552-0fcf278328ff None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:13.779368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74f162-9be8-4b99-9552-0fcf278328ff None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:13.807489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:13.813176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac647c-0530-4cc7-9f39-8e39acb27f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:13.813773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac647c-0530-4cc7-9f39-8e39acb27f1c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:13.814053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac647c-0530-4cc7-9f39-8e39acb27f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:13.814324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac647c-0530-4cc7-9f39-8e39acb27f1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:13.815048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac647c-0530-4cc7-9f39-8e39acb27f1c None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=10ce125d46b545499de1fcf7072963e5) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:13.815353 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac647c-0530-4cc7-9f39-8e39acb27f1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:13.817382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac647c-0530-4cc7-9f39-8e39acb27f1c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:13.823077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 614/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:44:13 2022] GET /identity/v3/role_assignments?user.id=10ce125d46b545499de1fcf7072963e5&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:15.819293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df1e36-efaf-4040-b1bf-efcca63e9b85 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:15.819819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df1e36-efaf-4040-b1bf-efcca63e9b85 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:15.819819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df1e36-efaf-4040-b1bf-efcca63e9b85 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:15.821218 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 606/1220] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:44:15.827514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90a323-fb72-40ee-8460-73ab9c929e21 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:15.827985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90a323-fb72-40ee-8460-73ab9c929e21 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:15.828147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90a323-fb72-40ee-8460-73ab9c929e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:15.846318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0c90a323-fb72-40ee-8460-73ab9c929e21 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:44:15.848234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0c90a323-fb72-40ee-8460-73ab9c929e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:44:15.914782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 615/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:44:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:44:15.937800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df1e36-efaf-4040-b1bf-efcca63e9b85 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:16.037600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df1e36-efaf-4040-b1bf-efcca63e9b85 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:16.075107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e52fc-8460-435f-b035-a4242de56af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:16.075107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e52fc-8460-435f-b035-a4242de56af0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:16.075107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e52fc-8460-435f-b035-a4242de56af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:16.075107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e52fc-8460-435f-b035-a4242de56af0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:16.084250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e52fc-8460-435f-b035-a4242de56af0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6c4e52fc-8460-435f-b035-a4242de56af0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:44:16.088985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:16.090168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:16.091177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 607/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:44:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:16.096258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90a323-fb72-40ee-8460-73ab9c929e21 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:16.193163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90a323-fb72-40ee-8460-73ab9c929e21 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:16.216923 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:16.227291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38d58f-e4a5-447b-a73f-7dd080694006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:16.227974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38d58f-e4a5-447b-a73f-7dd080694006 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:16.228249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38d58f-e4a5-447b-a73f-7dd080694006 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:16.228517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38d58f-e4a5-447b-a73f-7dd080694006 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:16.229208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38d58f-e4a5-447b-a73f-7dd080694006 None admin] RBAC: Adding query filter params (name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:16.229506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38d58f-e4a5-447b-a73f-7dd080694006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:16.230890 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38d58f-e4a5-447b-a73f-7dd080694006 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:16.239177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 616/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:44:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:16.245098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e52fc-8460-435f-b035-a4242de56af0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:16.288995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:16.344549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e52fc-8460-435f-b035-a4242de56af0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:16.379965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab739196-d7e1-4561-8091-80a11fcbbaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:16.380584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab739196-d7e1-4561-8091-80a11fcbbaef None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:16.380859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab739196-d7e1-4561-8091-80a11fcbbaef None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:16.381160 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab739196-d7e1-4561-8091-80a11fcbbaef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:16.391335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab739196-d7e1-4561-8091-80a11fcbbaef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:16.392758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab739196-d7e1-4561-8091-80a11fcbbaef None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ab739196-d7e1-4561-8091-80a11fcbbaef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:16.407358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:16.408354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:16.409540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:16.409540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:16.409540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:16.409540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 608/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:16.414273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38d58f-e4a5-447b-a73f-7dd080694006 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:16.513919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38d58f-e4a5-447b-a73f-7dd080694006 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:16.547275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14153c-ddd3-4a00-b1a9-480d2ff06abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:16.547870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14153c-ddd3-4a00-b1a9-480d2ff06abe None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:16.548144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14153c-ddd3-4a00-b1a9-480d2ff06abe None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:16.548413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14153c-ddd3-4a00-b1a9-480d2ff06abe None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:16.549068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14153c-ddd3-4a00-b1a9-480d2ff06abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:16.549377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14153c-ddd3-4a00-b1a9-480d2ff06abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:16.551406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14153c-ddd3-4a00-b1a9-480d2ff06abe None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:16.563110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 617/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:16.570013 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab739196-d7e1-4561-8091-80a11fcbbaef None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:16.662324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:16.668952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab739196-d7e1-4561-8091-80a11fcbbaef None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:16.702370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e53556-083b-4e64-b6ea-4ad0b8aefcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:16.702992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e53556-083b-4e64-b6ea-4ad0b8aefcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:16.703305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e53556-083b-4e64-b6ea-4ad0b8aefcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:16.703588 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e53556-083b-4e64-b6ea-4ad0b8aefcc5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:16.710589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e53556-083b-4e64-b6ea-4ad0b8aefcc5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:16.711997 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e53556-083b-4e64-b6ea-4ad0b8aefcc5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-15e53556-083b-4e64-b6ea-4ad0b8aefcc5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:44:16.723942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:16.724869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:16.725879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:16.725879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:16.725879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:44:16.725879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:44:16.725879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:44:16.725879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:44:16.725879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:44:16.725879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:44:16.725879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:16.725879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:16.725879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 609/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:44:16.730989 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14153c-ddd3-4a00-b1a9-480d2ff06abe None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:16.777365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:16.830096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14153c-ddd3-4a00-b1a9-480d2ff06abe None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:16.862754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7469375-a67f-4e7a-887d-c3fb3b00c523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:16.863961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7469375-a67f-4e7a-887d-c3fb3b00c523 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:16.864232 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7469375-a67f-4e7a-887d-c3fb3b00c523 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:16.864500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7469375-a67f-4e7a-887d-c3fb3b00c523 None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:16.865246 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7469375-a67f-4e7a-887d-c3fb3b00c523 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:16.865542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7469375-a67f-4e7a-887d-c3fb3b00c523 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:16.874503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7469375-a67f-4e7a-887d-c3fb3b00c523 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:16.883827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 618/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Aug 11 09:44:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:16.889641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e53556-083b-4e64-b6ea-4ad0b8aefcc5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:16.989028 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e53556-083b-4e64-b6ea-4ad0b8aefcc5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:17.022407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f71a55-67c7-45d4-b9ab-c6e61380766f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:17.023030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f71a55-67c7-45d4-b9ab-c6e61380766f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:17.023310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f71a55-67c7-45d4-b9ab-c6e61380766f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:17.023581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f71a55-67c7-45d4-b9ab-c6e61380766f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:17.033865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f71a55-67c7-45d4-b9ab-c6e61380766f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:17.036351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f71a55-67c7-45d4-b9ab-c6e61380766f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-48f71a55-67c7-45d4-b9ab-c6e61380766f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:17.050566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:17.051684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:17.052774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:17.052774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:17.052774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:17.052774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 610/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:17.057669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7469375-a67f-4e7a-887d-c3fb3b00c523 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:17.153612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7469375-a67f-4e7a-887d-c3fb3b00c523 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:17.155912 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:17.187083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11121263-f5fb-464d-9e82-d351e8d19048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:17.187687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11121263-f5fb-464d-9e82-d351e8d19048 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:17.187967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11121263-f5fb-464d-9e82-d351e8d19048 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:17.188245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11121263-f5fb-464d-9e82-d351e8d19048 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:17.188887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11121263-f5fb-464d-9e82-d351e8d19048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:17.189216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11121263-f5fb-464d-9e82-d351e8d19048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:17.190615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11121263-f5fb-464d-9e82-d351e8d19048 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:17.202292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 619/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:17.208661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f71a55-67c7-45d4-b9ab-c6e61380766f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:17.218442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:17.306084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f71a55-67c7-45d4-b9ab-c6e61380766f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:17.340120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11beb29b-cd95-4288-ad6f-fa04efc6731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:17.340745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beb29b-cd95-4288-ad6f-fa04efc6731f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:17.341042 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beb29b-cd95-4288-ad6f-fa04efc6731f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:17.341327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beb29b-cd95-4288-ad6f-fa04efc6731f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:17.348668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11beb29b-cd95-4288-ad6f-fa04efc6731f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:17.350120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11beb29b-cd95-4288-ad6f-fa04efc6731f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-11beb29b-cd95-4288-ad6f-fa04efc6731f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:44:17.360947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:17.361985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:17.362872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:44:17.362872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:44:17.362872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:44:17.362872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:44:17.362872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:44:17.362872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:44:17.362872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:17.362872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:17.362872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 611/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:44:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:17.368115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11121263-f5fb-464d-9e82-d351e8d19048 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:17.468893 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11121263-f5fb-464d-9e82-d351e8d19048 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:17.500986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf7cc3-d9ef-44cb-b124-c6bcba58008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:17.501615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf7cc3-d9ef-44cb-b124-c6bcba58008b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:17.501894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf7cc3-d9ef-44cb-b124-c6bcba58008b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:17.502164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf7cc3-d9ef-44cb-b124-c6bcba58008b None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:17.502897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf7cc3-d9ef-44cb-b124-c6bcba58008b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:17.503182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf7cc3-d9ef-44cb-b124-c6bcba58008b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:17.505245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf7cc3-d9ef-44cb-b124-c6bcba58008b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:17.516895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 620/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:44:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:17.523540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11beb29b-cd95-4288-ad6f-fa04efc6731f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:17.589026 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:17.625789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11beb29b-cd95-4288-ad6f-fa04efc6731f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:17.660385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8e750-cf1f-443f-89f8-c5707e8c8cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:17.661178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8e750-cf1f-443f-89f8-c5707e8c8cfe None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:17.661462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8e750-cf1f-443f-89f8-c5707e8c8cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:17.661736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8e750-cf1f-443f-89f8-c5707e8c8cfe None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/10ce125d46b545499de1fcf7072963e5/roles/c6c1ea7ecf3d4e00a138f97d91b14f73` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:17.677722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8e750-cf1f-443f-89f8-c5707e8c8cfe None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=10ce125d46b545499de1fcf7072963e5, role_id=c6c1ea7ecf3d4e00a138f97d91b14f73)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:17.679181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8e750-cf1f-443f-89f8-c5707e8c8cfe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:17.778460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 612/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:44:17 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/10ce125d46b545499de1fcf7072963e5/roles/c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 09:44:19.751422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd745-a36a-431c-9f77-45339a0473c3 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:19.751964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd745-a36a-431c-9f77-45339a0473c3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:19.751964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd745-a36a-431c-9f77-45339a0473c3 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:19.753299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 621/1233] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:44:19.758502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82601ac-c25a-4eaf-8dd4-96c34f03a126 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:19.758923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82601ac-c25a-4eaf-8dd4-96c34f03a126 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:19.759054 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82601ac-c25a-4eaf-8dd4-96c34f03a126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:19.776863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d82601ac-c25a-4eaf-8dd4-96c34f03a126 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:44:19.778125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d82601ac-c25a-4eaf-8dd4-96c34f03a126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:44:19.844326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 613/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:44:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:44:19.868482 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cd745-a36a-431c-9f77-45339a0473c3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:19.892710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:19.965565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cd745-a36a-431c-9f77-45339a0473c3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:19.998270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb331e1-5372-4e3e-859a-ed5d47de1ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:19.998879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb331e1-5372-4e3e-859a-ed5d47de1ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:19.999154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb331e1-5372-4e3e-859a-ed5d47de1ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:19.999422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb331e1-5372-4e3e-859a-ed5d47de1ee2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:20.009430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb331e1-5372-4e3e-859a-ed5d47de1ee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-5fb331e1-5372-4e3e-859a-ed5d47de1ee2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:44:20.014035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:20.015140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:20.015995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 622/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Aug 11 09:44:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 09:44:20.021874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82601ac-c25a-4eaf-8dd4-96c34f03a126 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:20.067065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:20.118207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82601ac-c25a-4eaf-8dd4-96c34f03a126 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:20.150578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717c656-e91d-490e-bd55-e27828e6e381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:20.151179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717c656-e91d-490e-bd55-e27828e6e381 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:20.151460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717c656-e91d-490e-bd55-e27828e6e381 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:20.151736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717c656-e91d-490e-bd55-e27828e6e381 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:20.152417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717c656-e91d-490e-bd55-e27828e6e381 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:20.152718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717c656-e91d-490e-bd55-e27828e6e381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:20.155047 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717c656-e91d-490e-bd55-e27828e6e381 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:20.163417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 614/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Aug 11 09:44:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:20.169179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb331e1-5372-4e3e-859a-ed5d47de1ee2 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:20.267134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb331e1-5372-4e3e-859a-ed5d47de1ee2 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:20.270229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:20.302998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423461f8-5b1f-4d3f-8ac6-c092b218ef7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:20.303614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423461f8-5b1f-4d3f-8ac6-c092b218ef7c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:20.303920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423461f8-5b1f-4d3f-8ac6-c092b218ef7c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:20.304194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423461f8-5b1f-4d3f-8ac6-c092b218ef7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:20.314138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423461f8-5b1f-4d3f-8ac6-c092b218ef7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:20.315598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423461f8-5b1f-4d3f-8ac6-c092b218ef7c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-423461f8-5b1f-4d3f-8ac6-c092b218ef7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:20.330439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:20.331559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:20.333110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:20.333110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:20.333110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:20.333110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 623/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:20.340464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717c656-e91d-490e-bd55-e27828e6e381 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:20.438405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717c656-e91d-490e-bd55-e27828e6e381 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:20.472357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186a471-fb61-4239-aa5b-1e6add847f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:20.472973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186a471-fb61-4239-aa5b-1e6add847f5b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:20.473281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186a471-fb61-4239-aa5b-1e6add847f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:20.473551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186a471-fb61-4239-aa5b-1e6add847f5b None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:20.474227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186a471-fb61-4239-aa5b-1e6add847f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:20.474607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186a471-fb61-4239-aa5b-1e6add847f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:20.476047 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186a471-fb61-4239-aa5b-1e6add847f5b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:20.487374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 615/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:20.493311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423461f8-5b1f-4d3f-8ac6-c092b218ef7c None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:20.512125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:20.592062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423461f8-5b1f-4d3f-8ac6-c092b218ef7c None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:20.625221 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af63a12-2869-4541-a282-201fc951fcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:20.625837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af63a12-2869-4541-a282-201fc951fcdb None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:20.626111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af63a12-2869-4541-a282-201fc951fcdb None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:20.626378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af63a12-2869-4541-a282-201fc951fcdb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:20.629530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:20.634265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af63a12-2869-4541-a282-201fc951fcdb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:20.635727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af63a12-2869-4541-a282-201fc951fcdb None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-7af63a12-2869-4541-a282-201fc951fcdb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:44:20.647101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:20.648216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:20.649373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:20.649373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:20.649373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:44:20.649373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:44:20.649373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:44:20.649373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:44:20.649373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:44:20.649373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:44:20.649373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:20.649373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:20.649373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 624/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:20.654143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186a471-fb61-4239-aa5b-1e6add847f5b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:20.750798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186a471-fb61-4239-aa5b-1e6add847f5b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:20.784611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9904e88-27db-453d-917a-dcc52776427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:20.785315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9904e88-27db-453d-917a-dcc52776427f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:20.785741 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9904e88-27db-453d-917a-dcc52776427f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:20.786014 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9904e88-27db-453d-917a-dcc52776427f None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:20.786738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9904e88-27db-453d-917a-dcc52776427f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:20.787044 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9904e88-27db-453d-917a-dcc52776427f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:20.788436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9904e88-27db-453d-917a-dcc52776427f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:20.805831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 616/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Aug 11 09:44:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:20.811534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af63a12-2869-4541-a282-201fc951fcdb None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:20.909711 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af63a12-2869-4541-a282-201fc951fcdb None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:20.942622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38b694-4fea-41a2-8d4c-1d571fd0cf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:20.943473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b694-4fea-41a2-8d4c-1d571fd0cf81 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:20.943748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b694-4fea-41a2-8d4c-1d571fd0cf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:20.944014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38b694-4fea-41a2-8d4c-1d571fd0cf81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:21.007996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:21.014374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38b694-4fea-41a2-8d4c-1d571fd0cf81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:21.015819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce38b694-4fea-41a2-8d4c-1d571fd0cf81 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:21.112013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 625/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:44:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:44:21.118803 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9904e88-27db-453d-917a-dcc52776427f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:21.145159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:21.213915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9904e88-27db-453d-917a-dcc52776427f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:21.246764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d463903-83a2-4147-9fbf-6fc52101fd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:21.247446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d463903-83a2-4147-9fbf-6fc52101fd74 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:21.247718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d463903-83a2-4147-9fbf-6fc52101fd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:21.247989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d463903-83a2-4147-9fbf-6fc52101fd74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:21.257858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d463903-83a2-4147-9fbf-6fc52101fd74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:21.259475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d463903-83a2-4147-9fbf-6fc52101fd74 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5d463903-83a2-4147-9fbf-6fc52101fd74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:21.273777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:21.274892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:21.275848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:21.275848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:21.275848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:21.275848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 617/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:21.280571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38b694-4fea-41a2-8d4c-1d571fd0cf81 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:21.372235 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:21.377783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce38b694-4fea-41a2-8d4c-1d571fd0cf81 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:21.409542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b58d5-ba82-48bd-ad5a-cc9119fea3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:21.410816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b58d5-ba82-48bd-ad5a-cc9119fea3ae None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:21.411097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b58d5-ba82-48bd-ad5a-cc9119fea3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:21.411363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b58d5-ba82-48bd-ad5a-cc9119fea3ae None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:21.411993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b58d5-ba82-48bd-ad5a-cc9119fea3ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:21.412288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b58d5-ba82-48bd-ad5a-cc9119fea3ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:21.413657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b58d5-ba82-48bd-ad5a-cc9119fea3ae None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:21.424376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 626/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:21.430635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d463903-83a2-4147-9fbf-6fc52101fd74 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:21.508547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:21.530163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d463903-83a2-4147-9fbf-6fc52101fd74 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:21.563435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdca4e6-07f2-4b6d-8a2a-09a523b0ef04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:21.564054 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdca4e6-07f2-4b6d-8a2a-09a523b0ef04 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:21.564336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdca4e6-07f2-4b6d-8a2a-09a523b0ef04 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:21.564609 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdca4e6-07f2-4b6d-8a2a-09a523b0ef04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:21.572304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdca4e6-07f2-4b6d-8a2a-09a523b0ef04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:21.573969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdca4e6-07f2-4b6d-8a2a-09a523b0ef04 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5bdca4e6-07f2-4b6d-8a2a-09a523b0ef04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:44:21.584888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:21.586005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:21.587097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:44:21.587097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:44:21.587097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:44:21.587097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:44:21.587097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:44:21.587097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:44:21.587097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:21.587097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:21.587097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 618/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:44:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:21.591864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b58d5-ba82-48bd-ad5a-cc9119fea3ae None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:21.689366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b58d5-ba82-48bd-ad5a-cc9119fea3ae None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:21.722244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f8ceb-6ac3-4b40-b9b9-2b64d4865f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:21.722881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f8ceb-6ac3-4b40-b9b9-2b64d4865f11 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:21.723156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f8ceb-6ac3-4b40-b9b9-2b64d4865f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:21.723426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f8ceb-6ac3-4b40-b9b9-2b64d4865f11 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:21.724104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f8ceb-6ac3-4b40-b9b9-2b64d4865f11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:21.724404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f8ceb-6ac3-4b40-b9b9-2b64d4865f11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:21.725840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f8ceb-6ac3-4b40-b9b9-2b64d4865f11 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:21.737294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 627/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:44:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:21.744084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdca4e6-07f2-4b6d-8a2a-09a523b0ef04 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:21.841867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdca4e6-07f2-4b6d-8a2a-09a523b0ef04 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:21.875605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ded1c-0e46-47e1-9ca3-14e8f1b46d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:21.876208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ded1c-0e46-47e1-9ca3-14e8f1b46d5f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:21.876487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ded1c-0e46-47e1-9ca3-14e8f1b46d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:21.876759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ded1c-0e46-47e1-9ca3-14e8f1b46d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:21.877518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ded1c-0e46-47e1-9ca3-14e8f1b46d5f None admin] RBAC: Adding query filter params (role.id=c6c1ea7ecf3d4e00a138f97d91b14f73, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=10ce125d46b545499de1fcf7072963e5) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:21.877826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ded1c-0e46-47e1-9ca3-14e8f1b46d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:21.879496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ded1c-0e46-47e1-9ca3-14e8f1b46d5f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:21.885162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 619/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:44:21 2022] GET /identity/v3/role_assignments?user.id=10ce125d46b545499de1fcf7072963e5&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=c6c1ea7ecf3d4e00a138f97d91b14f73 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:23.951718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748da9a-96af-4967-bcc7-64be1ebf748d None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:23.952241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748da9a-96af-4967-bcc7-64be1ebf748d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:23.952241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748da9a-96af-4967-bcc7-64be1ebf748d None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:23.953595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 628/1247] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:44:23.959287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822311f4-e241-4bd3-9eb7-f9d6b42fd417 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:23.959706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822311f4-e241-4bd3-9eb7-f9d6b42fd417 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:23.959867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822311f4-e241-4bd3-9eb7-f9d6b42fd417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:23.966189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:23.978678 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-822311f4-e241-4bd3-9eb7-f9d6b42fd417 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:44:23.979867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-822311f4-e241-4bd3-9eb7-f9d6b42fd417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:44:24.044542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 620/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:44:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:44:24.067593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748da9a-96af-4967-bcc7-64be1ebf748d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:24.086055 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:24.168557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748da9a-96af-4967-bcc7-64be1ebf748d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:24.206117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b49e21-da66-4253-8ae7-0b21cc6b0c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:24.206117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b49e21-da66-4253-8ae7-0b21cc6b0c77 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:24.206117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b49e21-da66-4253-8ae7-0b21cc6b0c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:24.206117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b49e21-da66-4253-8ae7-0b21cc6b0c77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:24.215115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b49e21-da66-4253-8ae7-0b21cc6b0c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-60b49e21-da66-4253-8ae7-0b21cc6b0c77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:44:24.219889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:24.220945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:24.222106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 629/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:44:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:24.227083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822311f4-e241-4bd3-9eb7-f9d6b42fd417 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:24.323745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822311f4-e241-4bd3-9eb7-f9d6b42fd417 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:24.352811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:24.357569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34477538-af4f-49f3-a72f-86cb176c6331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:24.358169 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34477538-af4f-49f3-a72f-86cb176c6331 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:24.358448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34477538-af4f-49f3-a72f-86cb176c6331 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:24.358717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34477538-af4f-49f3-a72f-86cb176c6331 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:24.359372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34477538-af4f-49f3-a72f-86cb176c6331 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:24.359672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34477538-af4f-49f3-a72f-86cb176c6331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:24.361039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34477538-af4f-49f3-a72f-86cb176c6331 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:24.368885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 621/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:44:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:44:24.374527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b49e21-da66-4253-8ae7-0b21cc6b0c77 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:24.452841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:24.477958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b49e21-da66-4253-8ae7-0b21cc6b0c77 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:24.508244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdec90-578a-4777-a60f-ea441d89c492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:24.508854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdec90-578a-4777-a60f-ea441d89c492 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:24.516146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdec90-578a-4777-a60f-ea441d89c492 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:24.516146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdec90-578a-4777-a60f-ea441d89c492 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:24.519065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdec90-578a-4777-a60f-ea441d89c492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:24.520546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdec90-578a-4777-a60f-ea441d89c492 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-f9fdec90-578a-4777-a60f-ea441d89c492 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:24.534668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:24.535749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:24.536618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:24.536618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:24.536618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:24.536618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 630/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:24.544022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34477538-af4f-49f3-a72f-86cb176c6331 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:24.643426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34477538-af4f-49f3-a72f-86cb176c6331 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:24.680713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ef895-c152-4025-bd55-b05ca8199f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:24.681349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ef895-c152-4025-bd55-b05ca8199f31 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:24.681634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ef895-c152-4025-bd55-b05ca8199f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:24.681914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ef895-c152-4025-bd55-b05ca8199f31 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:24.682546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ef895-c152-4025-bd55-b05ca8199f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:24.682848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ef895-c152-4025-bd55-b05ca8199f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:24.684186 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ef895-c152-4025-bd55-b05ca8199f31 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:24.696507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 622/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:24.702322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdec90-578a-4777-a60f-ea441d89c492 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:24.798826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdec90-578a-4777-a60f-ea441d89c492 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:24.824360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:24.832002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177716a-fb4d-4fde-882e-91cbf097eb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:24.832609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177716a-fb4d-4fde-882e-91cbf097eb3e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:24.832885 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177716a-fb4d-4fde-882e-91cbf097eb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:24.833197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177716a-fb4d-4fde-882e-91cbf097eb3e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:24.840266 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177716a-fb4d-4fde-882e-91cbf097eb3e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:24.841710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177716a-fb4d-4fde-882e-91cbf097eb3e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-c177716a-fb4d-4fde-882e-91cbf097eb3e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:44:24.852758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:24.854553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:24.856317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:24.856317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:24.856317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:44:24.856317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:44:24.856317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:44:24.856317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:44:24.856317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:44:24.856317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:44:24.856317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:24.856317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:24.856317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 631/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:24.859771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ef895-c152-4025-bd55-b05ca8199f31 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:24.913901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:24.957384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ef895-c152-4025-bd55-b05ca8199f31 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:24.991460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c06d0-8a4d-41b8-8cd2-41fe5a26e5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:24.992068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c06d0-8a4d-41b8-8cd2-41fe5a26e5cf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:24.992345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c06d0-8a4d-41b8-8cd2-41fe5a26e5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:24.992616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c06d0-8a4d-41b8-8cd2-41fe5a26e5cf None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:24.993361 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c06d0-8a4d-41b8-8cd2-41fe5a26e5cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:24.993663 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c06d0-8a4d-41b8-8cd2-41fe5a26e5cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:24.995051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c06d0-8a4d-41b8-8cd2-41fe5a26e5cf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:25.012402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 623/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Aug 11 09:44:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:25.017881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177716a-fb4d-4fde-882e-91cbf097eb3e None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:25.116967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177716a-fb4d-4fde-882e-91cbf097eb3e None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:25.151695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2ced0-cc6e-44c1-bf6c-f9fd045f9d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:25.152296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2ced0-cc6e-44c1-bf6c-f9fd045f9d8f None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:25.152576 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2ced0-cc6e-44c1-bf6c-f9fd045f9d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:25.152848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2ced0-cc6e-44c1-bf6c-f9fd045f9d8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:25.207615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:25.219187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2ced0-cc6e-44c1-bf6c-f9fd045f9d8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:25.220627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2ced0-cc6e-44c1-bf6c-f9fd045f9d8f None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:25.314119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 632/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:44:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:44:25.321263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c06d0-8a4d-41b8-8cd2-41fe5a26e5cf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:25.418516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c06d0-8a4d-41b8-8cd2-41fe5a26e5cf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:25.431831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:25.452825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6d97d-8457-4eed-b70b-22bf3174ff68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:25.453524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6d97d-8457-4eed-b70b-22bf3174ff68 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:25.453803 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6d97d-8457-4eed-b70b-22bf3174ff68 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:25.454076 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6d97d-8457-4eed-b70b-22bf3174ff68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:25.463837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6d97d-8457-4eed-b70b-22bf3174ff68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:25.465415 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a6d97d-8457-4eed-b70b-22bf3174ff68 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-62a6d97d-8457-4eed-b70b-22bf3174ff68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:25.479359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:25.480470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:25.481569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:25.481569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:25.481569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:25.481569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 624/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:25.486249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2ced0-cc6e-44c1-bf6c-f9fd045f9d8f None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:25.572285 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:25.583831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2ced0-cc6e-44c1-bf6c-f9fd045f9d8f None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:25.617694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92347dea-c866-4927-b721-8dc58f793a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:25.618284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92347dea-c866-4927-b721-8dc58f793a05 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:25.618627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92347dea-c866-4927-b721-8dc58f793a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:25.618897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92347dea-c866-4927-b721-8dc58f793a05 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:25.619524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92347dea-c866-4927-b721-8dc58f793a05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:25.619819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92347dea-c866-4927-b721-8dc58f793a05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:25.621194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92347dea-c866-4927-b721-8dc58f793a05 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:25.632083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 633/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:25.638046 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6d97d-8457-4eed-b70b-22bf3174ff68 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:25.739540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6d97d-8457-4eed-b70b-22bf3174ff68 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:25.775324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1384db5-fe67-4bad-82d3-52040c56da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:25.775948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1384db5-fe67-4bad-82d3-52040c56da31 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:25.776224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1384db5-fe67-4bad-82d3-52040c56da31 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:25.776489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1384db5-fe67-4bad-82d3-52040c56da31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:25.782820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1384db5-fe67-4bad-82d3-52040c56da31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:25.784376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1384db5-fe67-4bad-82d3-52040c56da31 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f1384db5-fe67-4bad-82d3-52040c56da31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:44:25.795612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:25.796737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:25.797861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:44:25.797861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:44:25.797861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:44:25.797861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:44:25.797861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:44:25.797861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:44:25.797861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:25.797861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:25.797861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 625/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:44:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:25.803627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92347dea-c866-4927-b721-8dc58f793a05 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:25.902465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92347dea-c866-4927-b721-8dc58f793a05 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:25.935398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b916a-bace-4922-9dc5-f37bc6f02fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:25.935979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b916a-bace-4922-9dc5-f37bc6f02fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:25.936253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b916a-bace-4922-9dc5-f37bc6f02fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:25.936521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b916a-bace-4922-9dc5-f37bc6f02fd1 None admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:25.937223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b916a-bace-4922-9dc5-f37bc6f02fd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:25.937523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b916a-bace-4922-9dc5-f37bc6f02fd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:25.938931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b916a-bace-4922-9dc5-f37bc6f02fd1 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:25.951314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 634/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:44:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:25.957515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1384db5-fe67-4bad-82d3-52040c56da31 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:25.983665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:26.056083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1384db5-fe67-4bad-82d3-52040c56da31 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:26.090929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dada4a7-7cad-4055-bae9-4f41497a994c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:26.091535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dada4a7-7cad-4055-bae9-4f41497a994c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:26.091818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dada4a7-7cad-4055-bae9-4f41497a994c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:26.092094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dada4a7-7cad-4055-bae9-4f41497a994c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:26.092819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dada4a7-7cad-4055-bae9-4f41497a994c None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=10ce125d46b545499de1fcf7072963e5) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:26.093160 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dada4a7-7cad-4055-bae9-4f41497a994c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:26.094570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dada4a7-7cad-4055-bae9-4f41497a994c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:26.100554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 626/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:44:25 2022] GET /identity/v3/role_assignments?user.id=10ce125d46b545499de1fcf7072963e5&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:28.089934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f626a-8c5b-468b-a19f-f796f2207a55 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:28.090514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f626a-8c5b-468b-a19f-f796f2207a55 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:28.090514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f626a-8c5b-468b-a19f-f796f2207a55 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:28.091795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 635/1261] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:44:28.096989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b90e3f-862d-40a8-b4fe-b9650633bedf None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:28.097409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b90e3f-862d-40a8-b4fe-b9650633bedf None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:28.097571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b90e3f-862d-40a8-b4fe-b9650633bedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:28.116010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b2b90e3f-862d-40a8-b4fe-b9650633bedf None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:44:28.117277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b2b90e3f-862d-40a8-b4fe-b9650633bedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:44:28.184839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 627/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:44:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:44:28.207493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f626a-8c5b-468b-a19f-f796f2207a55 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:28.221030 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:28.308047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f626a-8c5b-468b-a19f-f796f2207a55 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:28.342271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59206d38-c067-4725-a0ca-a1d20a9c2de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:28.342820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59206d38-c067-4725-a0ca-a1d20a9c2de3 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:28.343094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59206d38-c067-4725-a0ca-a1d20a9c2de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:28.343361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59206d38-c067-4725-a0ca-a1d20a9c2de3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:28.353220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59206d38-c067-4725-a0ca-a1d20a9c2de3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-59206d38-c067-4725-a0ca-a1d20a9c2de3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:44:28.357957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:28.359119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:28.360013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 636/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:44:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:28.365858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b90e3f-862d-40a8-b4fe-b9650633bedf None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:28.384316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:28.462608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b90e3f-862d-40a8-b4fe-b9650633bedf None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:28.496754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11e6d8-29e2-4485-9292-6ee1aaf17ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:28.497392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11e6d8-29e2-4485-9292-6ee1aaf17ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:28.497680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11e6d8-29e2-4485-9292-6ee1aaf17ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:28.497957 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11e6d8-29e2-4485-9292-6ee1aaf17ec1 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:28.498619 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11e6d8-29e2-4485-9292-6ee1aaf17ec1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:28.498925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11e6d8-29e2-4485-9292-6ee1aaf17ec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:28.500273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11e6d8-29e2-4485-9292-6ee1aaf17ec1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:28.508262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 628/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:44:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:44:28.514867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59206d38-c067-4725-a0ca-a1d20a9c2de3 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:28.583331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:28.616058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59206d38-c067-4725-a0ca-a1d20a9c2de3 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:28.649561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4b25a-0a70-4572-9e4a-c85a2c164db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:28.659640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4b25a-0a70-4572-9e4a-c85a2c164db7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:28.659640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4b25a-0a70-4572-9e4a-c85a2c164db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:28.659640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4b25a-0a70-4572-9e4a-c85a2c164db7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:28.660537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4b25a-0a70-4572-9e4a-c85a2c164db7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:28.662513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4b25a-0a70-4572-9e4a-c85a2c164db7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-bda4b25a-0a70-4572-9e4a-c85a2c164db7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:28.676274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:28.678048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:28.678942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:28.678942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:28.678942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:28.678942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 637/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:28.683799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11e6d8-29e2-4485-9292-6ee1aaf17ec1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:28.765911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:28.784096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11e6d8-29e2-4485-9292-6ee1aaf17ec1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:28.818195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c3625-86db-42ef-b04a-a999b8c1268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:28.819098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c3625-86db-42ef-b04a-a999b8c1268f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:28.819379 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c3625-86db-42ef-b04a-a999b8c1268f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:28.819653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c3625-86db-42ef-b04a-a999b8c1268f None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:28.820288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c3625-86db-42ef-b04a-a999b8c1268f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:28.820589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c3625-86db-42ef-b04a-a999b8c1268f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:28.821962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c3625-86db-42ef-b04a-a999b8c1268f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:28.832847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 629/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:28.838604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4b25a-0a70-4572-9e4a-c85a2c164db7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:28.941558 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4b25a-0a70-4572-9e4a-c85a2c164db7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:28.962334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:28.975846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d9813-326d-433b-8798-e96db0e9633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:28.976462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9813-326d-433b-8798-e96db0e9633a None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:28.976742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9813-326d-433b-8798-e96db0e9633a None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:28.977040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9813-326d-433b-8798-e96db0e9633a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:28.984858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d9813-326d-433b-8798-e96db0e9633a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:28.990105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d9813-326d-433b-8798-e96db0e9633a None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-ae4d9813-326d-433b-8798-e96db0e9633a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:44:28.997570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:28.998688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:28.999898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:28.999898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:28.999898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:44:28.999898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:44:28.999898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:44:28.999898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:44:28.999898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:44:28.999898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:44:28.999898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:28.999898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:28.999898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 638/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:29.004711 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c3625-86db-42ef-b04a-a999b8c1268f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:29.103846 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c3625-86db-42ef-b04a-a999b8c1268f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:29.137200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebc1c5-56df-4b1e-8247-320643df1ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:29.137798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebc1c5-56df-4b1e-8247-320643df1ccb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:29.138072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebc1c5-56df-4b1e-8247-320643df1ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:29.138340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebc1c5-56df-4b1e-8247-320643df1ccb None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:29.139107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bebc1c5-56df-4b1e-8247-320643df1ccb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:29.139409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bebc1c5-56df-4b1e-8247-320643df1ccb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:29.140800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bebc1c5-56df-4b1e-8247-320643df1ccb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:29.149700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:29.157428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 630/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Aug 11 09:44:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:29.163256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d9813-326d-433b-8798-e96db0e9633a None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:29.263413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d9813-326d-433b-8798-e96db0e9633a None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:29.295947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1fa69-69cb-4ce4-a720-821aed5bcaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:29.296560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1fa69-69cb-4ce4-a720-821aed5bcaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:29.296838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1fa69-69cb-4ce4-a720-821aed5bcaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:29.297138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1fa69-69cb-4ce4-a720-821aed5bcaa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:29.307998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1fa69-69cb-4ce4-a720-821aed5bcaa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:29.309457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1fa69-69cb-4ce4-a720-821aed5bcaa7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-abe1fa69-69cb-4ce4-a720-821aed5bcaa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:29.323780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:29.324790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:29.325693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:29.325693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:29.325693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:29.325693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 639/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:29.330838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebc1c5-56df-4b1e-8247-320643df1ccb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:29.426888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebc1c5-56df-4b1e-8247-320643df1ccb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:29.461723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a382902-d087-49b1-80e6-13dfe92a0833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:29.462326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a382902-d087-49b1-80e6-13dfe92a0833 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:29.462602 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a382902-d087-49b1-80e6-13dfe92a0833 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:29.462868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a382902-d087-49b1-80e6-13dfe92a0833 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:29.463498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a382902-d087-49b1-80e6-13dfe92a0833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:29.463797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a382902-d087-49b1-80e6-13dfe92a0833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:29.465216 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a382902-d087-49b1-80e6-13dfe92a0833 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:29.476697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 631/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:29.482489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1fa69-69cb-4ce4-a720-821aed5bcaa7 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:29.513731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:29.579957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1fa69-69cb-4ce4-a720-821aed5bcaa7 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:29.612982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652f433-c445-4f0c-9699-033daf2eca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:29.620648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652f433-c445-4f0c-9699-033daf2eca6b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:29.620648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652f433-c445-4f0c-9699-033daf2eca6b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:29.620648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652f433-c445-4f0c-9699-033daf2eca6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:29.621180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652f433-c445-4f0c-9699-033daf2eca6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:29.622608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652f433-c445-4f0c-9699-033daf2eca6b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-b652f433-c445-4f0c-9699-033daf2eca6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:44:29.633693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:29.634736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:29.635802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:44:29.635802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:44:29.635802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:44:29.635802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:44:29.635802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:44:29.635802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:44:29.635802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:29.635802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:29.635802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 640/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:44:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:29.640631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a382902-d087-49b1-80e6-13dfe92a0833 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:29.700350 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:29.740730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a382902-d087-49b1-80e6-13dfe92a0833 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:29.773298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d927d-84ee-450a-af93-cbed982f5f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:29.775597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d927d-84ee-450a-af93-cbed982f5f0c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:29.775597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d927d-84ee-450a-af93-cbed982f5f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:29.775597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d927d-84ee-450a-af93-cbed982f5f0c None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:29.775597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d927d-84ee-450a-af93-cbed982f5f0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:29.775597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d927d-84ee-450a-af93-cbed982f5f0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:29.776744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d927d-84ee-450a-af93-cbed982f5f0c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:29.788853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 632/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:44:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:29.794910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652f433-c445-4f0c-9699-033daf2eca6b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:29.887040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:29.895553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652f433-c445-4f0c-9699-033daf2eca6b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:29.928923 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040e557-1fe3-4088-9e73-f5a0e3ada13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:29.929584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040e557-1fe3-4088-9e73-f5a0e3ada13b None admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:29.929863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040e557-1fe3-4088-9e73-f5a0e3ada13b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:29.930134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040e557-1fe3-4088-9e73-f5a0e3ada13b None admin] PATH_INFO: `/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/10ce125d46b545499de1fcf7072963e5/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:29.932015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040e557-1fe3-4088-9e73-f5a0e3ada13b None admin] RBAC: Authorizing `identity:create_grant(project_id=ca874f4c98ba4c499cf22ad790ac4c34, user_id=10ce125d46b545499de1fcf7072963e5, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:29.933517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040e557-1fe3-4088-9e73-f5a0e3ada13b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:30.043550 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 641/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Aug 11 09:44:29 2022] PUT /identity/v3/projects/ca874f4c98ba4c499cf22ad790ac4c34/users/10ce125d46b545499de1fcf7072963e5/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:44:32.034903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f4c32-413b-49ea-bf63-84a1f95c51ae None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:32.035429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f4c32-413b-49ea-bf63-84a1f95c51ae None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:32.035546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f4c32-413b-49ea-bf63-84a1f95c51ae None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:32.036838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 633/1274] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:44:32.045867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0f901-c3f1-4c06-b921-f5a7e4e8d529 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:32.045867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0f901-c3f1-4c06-b921-f5a7e4e8d529 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:32.045867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0f901-c3f1-4c06-b921-f5a7e4e8d529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:32.063004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-97b0f901-c3f1-4c06-b921-f5a7e4e8d529 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:44:32.064207 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-97b0f901-c3f1-4c06-b921-f5a7e4e8d529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:44:32.129502 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 642/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:44:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:44:32.152820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f4c32-413b-49ea-bf63-84a1f95c51ae None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:32.257230 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f4c32-413b-49ea-bf63-84a1f95c51ae None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:32.274344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:32.291250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11672ff9-96a4-484c-acfb-b21db9070cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:32.291818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11672ff9-96a4-484c-acfb-b21db9070cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:32.292095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11672ff9-96a4-484c-acfb-b21db9070cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:32.292363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11672ff9-96a4-484c-acfb-b21db9070cb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:32.302076 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11672ff9-96a4-484c-acfb-b21db9070cb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-11672ff9-96a4-484c-acfb-b21db9070cb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 09:44:32.308449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:32.309613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:32.310666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 634/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Aug 11 09:44:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:32.314671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0f901-c3f1-4c06-b921-f5a7e4e8d529 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:32.395467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:32.414366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0f901-c3f1-4c06-b921-f5a7e4e8d529 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:32.448092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3ab42-4ba8-4144-8a51-ccb7da76a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:32.448685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3ab42-4ba8-4144-8a51-ccb7da76a027 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:32.448966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3ab42-4ba8-4144-8a51-ccb7da76a027 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:32.449271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3ab42-4ba8-4144-8a51-ccb7da76a027 None admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:32.449931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3ab42-4ba8-4144-8a51-ccb7da76a027 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:32.450232 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3ab42-4ba8-4144-8a51-ccb7da76a027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:32.451589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3ab42-4ba8-4144-8a51-ccb7da76a027 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:32.459427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 643/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Aug 11 09:44:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:32.465458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11672ff9-96a4-484c-acfb-b21db9070cb9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:32.571393 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11672ff9-96a4-484c-acfb-b21db9070cb9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:32.605447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798c356-2ebb-46ed-a135-e9f94630aeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:32.606074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798c356-2ebb-46ed-a135-e9f94630aeea None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:32.606355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798c356-2ebb-46ed-a135-e9f94630aeea None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:32.606628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798c356-2ebb-46ed-a135-e9f94630aeea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:32.617243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798c356-2ebb-46ed-a135-e9f94630aeea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:32.618677 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798c356-2ebb-46ed-a135-e9f94630aeea None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-7798c356-2ebb-46ed-a135-e9f94630aeea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:32.632703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:32.642688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:32.643608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:32.643608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:32.643608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:32.643608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 635/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:32.643906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3ab42-4ba8-4144-8a51-ccb7da76a027 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:32.736941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3ab42-4ba8-4144-8a51-ccb7da76a027 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:32.768462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed8003-6621-468e-9ad7-e0beab306e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:32.769080 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed8003-6621-468e-9ad7-e0beab306e76 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:32.769358 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed8003-6621-468e-9ad7-e0beab306e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:32.769624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed8003-6621-468e-9ad7-e0beab306e76 None admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:32.770256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed8003-6621-468e-9ad7-e0beab306e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:32.770552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed8003-6621-468e-9ad7-e0beab306e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:32.772400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed8003-6621-468e-9ad7-e0beab306e76 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:32.783426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 644/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:32.789821 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798c356-2ebb-46ed-a135-e9f94630aeea None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:32.820440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:32.888510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798c356-2ebb-46ed-a135-e9f94630aeea None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:32.924911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00678b6e-779e-4f37-b0f8-d3bb3c461a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:32.925562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00678b6e-779e-4f37-b0f8-d3bb3c461a41 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:32.925840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00678b6e-779e-4f37-b0f8-d3bb3c461a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:32.926110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00678b6e-779e-4f37-b0f8-d3bb3c461a41 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:32.933111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00678b6e-779e-4f37-b0f8-d3bb3c461a41 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:32.934463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00678b6e-779e-4f37-b0f8-d3bb3c461a41 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-00678b6e-779e-4f37-b0f8-d3bb3c461a41 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 09:44:32.946180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:32.950303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:32.951520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:32.951520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:32.951520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 09:44:32.951520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 09:44:32.951520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 09:44:32.951520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 09:44:32.951520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 09:44:32.951520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 09:44:32.951520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 09:44:32.951520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:32.951520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 636/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:32.953338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed8003-6621-468e-9ad7-e0beab306e76 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:32.957265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:33.051203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed8003-6621-468e-9ad7-e0beab306e76 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:33.085654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149edc6-4229-49b6-a022-8e591ea3ef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:33.086259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149edc6-4229-49b6-a022-8e591ea3ef9b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:33.086541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149edc6-4229-49b6-a022-8e591ea3ef9b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:33.086814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149edc6-4229-49b6-a022-8e591ea3ef9b None admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:33.087587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149edc6-4229-49b6-a022-8e591ea3ef9b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:33.087896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149edc6-4229-49b6-a022-8e591ea3ef9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:33.089425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149edc6-4229-49b6-a022-8e591ea3ef9b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:33.106370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 645/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Aug 11 09:44:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:33.112462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00678b6e-779e-4f37-b0f8-d3bb3c461a41 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:33.202002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:33.212029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00678b6e-779e-4f37-b0f8-d3bb3c461a41 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:33.244712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030bae1-fcb2-4707-b35a-448c20d3e9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:33.245351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030bae1-fcb2-4707-b35a-448c20d3e9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:33.245635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030bae1-fcb2-4707-b35a-448c20d3e9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:33.245908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030bae1-fcb2-4707-b35a-448c20d3e9e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:33.311197 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030bae1-fcb2-4707-b35a-448c20d3e9e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:33.312630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030bae1-fcb2-4707-b35a-448c20d3e9e0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:33.403208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:33.413157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 637/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Aug 11 09:44:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 09:44:33.418877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149edc6-4229-49b6-a022-8e591ea3ef9b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:33.478072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:33.514854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149edc6-4229-49b6-a022-8e591ea3ef9b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:33.547783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb3b93-f5d1-4369-9821-ad3364032122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:33.548414 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb3b93-f5d1-4369-9821-ad3364032122 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:33.548692 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb3b93-f5d1-4369-9821-ad3364032122 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:33.548965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb3b93-f5d1-4369-9821-ad3364032122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:33.560045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb3b93-f5d1-4369-9821-ad3364032122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:33.568560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cb3b93-f5d1-4369-9821-ad3364032122 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-33cb3b93-f5d1-4369-9821-ad3364032122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 09:44:33.576930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:33.578068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 09:44:33.579365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 09:44:33.579365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 09:44:33.579365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:33.579365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 646/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:44:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:33.583755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030bae1-fcb2-4707-b35a-448c20d3e9e0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:33.680517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030bae1-fcb2-4707-b35a-448c20d3e9e0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:33.713994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb47b2a-3bbb-49a2-b05e-7399e87c4ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:33.714589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb47b2a-3bbb-49a2-b05e-7399e87c4ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:33.714865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb47b2a-3bbb-49a2-b05e-7399e87c4ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:33.715136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb47b2a-3bbb-49a2-b05e-7399e87c4ec6 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:33.715764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb47b2a-3bbb-49a2-b05e-7399e87c4ec6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:33.716062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb47b2a-3bbb-49a2-b05e-7399e87c4ec6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:33.717436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb47b2a-3bbb-49a2-b05e-7399e87c4ec6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:33.728349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 638/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Aug 11 09:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:33.733970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb3b93-f5d1-4369-9821-ad3364032122 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:33.832735 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb3b93-f5d1-4369-9821-ad3364032122 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:33.846423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:33.866857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b09d9-f589-4574-89c4-625734d83f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:33.867481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b09d9-f589-4574-89c4-625734d83f9b None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:33.867759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b09d9-f589-4574-89c4-625734d83f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:33.868030 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b09d9-f589-4574-89c4-625734d83f9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:33.874434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b09d9-f589-4574-89c4-625734d83f9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:33.875895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b09d9-f589-4574-89c4-625734d83f9b None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application [None req-f11b09d9-f589-4574-89c4-625734d83f9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 09:44:33.887649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:33.888780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:33.891908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 09:44:33.891908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 09:44:33.891908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 09:44:33.891908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 09:44:33.891908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 09:44:33.891908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 09:44:33.891908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 09:44:33.891908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: ERROR keystone.server.flask.application Aug 11 09:44:33.891908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 647/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Aug 11 09:44:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:33.894974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb47b2a-3bbb-49a2-b05e-7399e87c4ec6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:33.945972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:33.996602 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb47b2a-3bbb-49a2-b05e-7399e87c4ec6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:34.028418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f654b-7100-4d3b-8a15-25a8b93fc795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:34.029048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f654b-7100-4d3b-8a15-25a8b93fc795 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:34.029332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f654b-7100-4d3b-8a15-25a8b93fc795 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:34.029606 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f654b-7100-4d3b-8a15-25a8b93fc795 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:34.030297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f654b-7100-4d3b-8a15-25a8b93fc795 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:34.030596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f654b-7100-4d3b-8a15-25a8b93fc795 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:34.031997 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f654b-7100-4d3b-8a15-25a8b93fc795 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:34.044000 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 639/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Aug 11 09:44:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:34.050850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b09d9-f589-4574-89c4-625734d83f9b None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:34.155621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b09d9-f589-4574-89c4-625734d83f9b None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:34.190855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84517d-cb37-4d59-8468-7c6f8c1bc827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:34.191419 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84517d-cb37-4d59-8468-7c6f8c1bc827 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:34.191694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84517d-cb37-4d59-8468-7c6f8c1bc827 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:34.191961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84517d-cb37-4d59-8468-7c6f8c1bc827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:34.192684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84517d-cb37-4d59-8468-7c6f8c1bc827 None admin] RBAC: Adding query filter params (role.id=caa057fb0c7c45258fbfb9dee30a22b3, scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34, user.id=10ce125d46b545499de1fcf7072963e5) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:34.192987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84517d-cb37-4d59-8468-7c6f8c1bc827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:34.194892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84517d-cb37-4d59-8468-7c6f8c1bc827 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:34.200584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 648/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Aug 11 09:44:34 2022] GET /identity/v3/role_assignments?user.id=10ce125d46b545499de1fcf7072963e5&scope.project.id=ca874f4c98ba4c499cf22ad790ac4c34&role.id=caa057fb0c7c45258fbfb9dee30a22b3 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:36.227447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965c4ba-f497-4740-8ced-e5c96ac16303 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:36.227975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965c4ba-f497-4740-8ced-e5c96ac16303 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:36.227975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965c4ba-f497-4740-8ced-e5c96ac16303 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:36.229360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 640/1288] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:44:36.234719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bda516-fa05-431c-b2f0-de77e2e9e891 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:36.235179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bda516-fa05-431c-b2f0-de77e2e9e891 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:36.235281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bda516-fa05-431c-b2f0-de77e2e9e891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:36.254094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-51bda516-fa05-431c-b2f0-de77e2e9e891 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:44:36.255288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-51bda516-fa05-431c-b2f0-de77e2e9e891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:44:36.284334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:36.321732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 649/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:44:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:44:36.344188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965c4ba-f497-4740-8ced-e5c96ac16303 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:36.445256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965c4ba-f497-4740-8ced-e5c96ac16303 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:36.455557 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:36.479417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ebf79-ee9d-4d1b-97e8-773c3a3e792c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:36.480039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ebf79-ee9d-4d1b-97e8-773c3a3e792c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:36.480320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ebf79-ee9d-4d1b-97e8-773c3a3e792c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:36.480591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ebf79-ee9d-4d1b-97e8-773c3a3e792c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:36.481203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ebf79-ee9d-4d1b-97e8-773c3a3e792c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:36.482621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ebf79-ee9d-4d1b-97e8-773c3a3e792c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6a5ebf79-ee9d-4d1b-97e8-773c3a3e792c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:36.491027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:36.492224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:36.493161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 641/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Aug 11 09:44:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:36.498250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bda516-fa05-431c-b2f0-de77e2e9e891 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:36.595522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bda516-fa05-431c-b2f0-de77e2e9e891 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:36.629084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac827653-5c0d-403d-bbd0-d0cfa01435c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:36.629683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac827653-5c0d-403d-bbd0-d0cfa01435c8 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:36.629965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac827653-5c0d-403d-bbd0-d0cfa01435c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:36.630319 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac827653-5c0d-403d-bbd0-d0cfa01435c8 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:36.630953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac827653-5c0d-403d-bbd0-d0cfa01435c8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:36.631267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac827653-5c0d-403d-bbd0-d0cfa01435c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:36.632964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac827653-5c0d-403d-bbd0-d0cfa01435c8 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:36.637679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 650/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Aug 11 09:44:36 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:36.643609 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ebf79-ee9d-4d1b-97e8-773c3a3e792c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:36.744496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ebf79-ee9d-4d1b-97e8-773c3a3e792c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:36.777763 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4fbc1-5953-4839-baae-92c0e001020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:36.778402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4fbc1-5953-4839-baae-92c0e001020f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:36.778682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4fbc1-5953-4839-baae-92c0e001020f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:36.778952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4fbc1-5953-4839-baae-92c0e001020f None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:36.779621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4fbc1-5953-4839-baae-92c0e001020f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:36.779921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4fbc1-5953-4839-baae-92c0e001020f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:36.781277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4fbc1-5953-4839-baae-92c0e001020f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:36.785550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 642/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Aug 11 09:44:36 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:38.775389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011a23-6412-461e-a31c-2adf3ca64cfe None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:38.775921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011a23-6412-461e-a31c-2adf3ca64cfe None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:38.775921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011a23-6412-461e-a31c-2adf3ca64cfe None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:38.777273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 651/1293] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:44:38.782485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5aa81-ff0a-438e-a5a2-bcc3c4c54ae0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:38.782880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5aa81-ff0a-438e-a5a2-bcc3c4c54ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:38.783039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5aa81-ff0a-438e-a5a2-bcc3c4c54ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:38.801193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3cf5aa81-ff0a-438e-a5a2-bcc3c4c54ae0 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:44:38.802466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3cf5aa81-ff0a-438e-a5a2-bcc3c4c54ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:44:38.832726 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:38.868812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 643/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:44:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:44:38.891723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd011a23-6412-461e-a31c-2adf3ca64cfe None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:38.922585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:38.989911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd011a23-6412-461e-a31c-2adf3ca64cfe None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:39.023497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ca52f-7c3e-4c5a-bb47-90fcc3ccf9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:39.024090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca52f-7c3e-4c5a-bb47-90fcc3ccf9a7 None admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:39.024370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca52f-7c3e-4c5a-bb47-90fcc3ccf9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:39.024641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca52f-7c3e-4c5a-bb47-90fcc3ccf9a7 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:39.025303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ca52f-7c3e-4c5a-bb47-90fcc3ccf9a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:39.026706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ca52f-7c3e-4c5a-bb47-90fcc3ccf9a7 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:39.106614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 652/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Aug 11 09:44:38 2022] POST /identity/v3/services => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:44:41.122427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498ecd7-4c6d-409c-9ed8-7c9a84458cf6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:41.123007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498ecd7-4c6d-409c-9ed8-7c9a84458cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:41.123007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498ecd7-4c6d-409c-9ed8-7c9a84458cf6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:41.124321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 644/1296] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:44:41.129435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac3794-e4a5-4339-9bf9-322f274b26dd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:41.129836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac3794-e4a5-4339-9bf9-322f274b26dd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:41.130014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac3794-e4a5-4339-9bf9-322f274b26dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:41.148706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-35ac3794-e4a5-4339-9bf9-322f274b26dd None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:44:41.149964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-35ac3794-e4a5-4339-9bf9-322f274b26dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:44:41.223349 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 653/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:44:41 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:44:41.246051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498ecd7-4c6d-409c-9ed8-7c9a84458cf6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:41.345447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498ecd7-4c6d-409c-9ed8-7c9a84458cf6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:41.380284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cda78-8a17-4420-ad7a-5d8be8944691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:41.380852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cda78-8a17-4420-ad7a-5d8be8944691 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:41.381162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cda78-8a17-4420-ad7a-5d8be8944691 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:41.381529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cda78-8a17-4420-ad7a-5d8be8944691 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:41.382320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cda78-8a17-4420-ad7a-5d8be8944691 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:41.383665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cda78-8a17-4420-ad7a-5d8be8944691 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5c7cda78-8a17-4420-ad7a-5d8be8944691 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:41.392303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:41.396217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:41.397318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 645/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Aug 11 09:44:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:41.399571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac3794-e4a5-4339-9bf9-322f274b26dd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:41.418481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:41.499439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac3794-e4a5-4339-9bf9-322f274b26dd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:41.532925 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e9009-cfd9-412b-bd63-a8ffa67a1ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:41.533542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e9009-cfd9-412b-bd63-a8ffa67a1ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:41.533818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e9009-cfd9-412b-bd63-a8ffa67a1ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:41.534087 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e9009-cfd9-412b-bd63-a8ffa67a1ac9 None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:41.534715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e9009-cfd9-412b-bd63-a8ffa67a1ac9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:41.535013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e9009-cfd9-412b-bd63-a8ffa67a1ac9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:41.536404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46e9009-cfd9-412b-bd63-a8ffa67a1ac9 None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:41.542001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 654/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Aug 11 09:44:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:41.548028 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cda78-8a17-4420-ad7a-5d8be8944691 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:41.580064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:41.647278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cda78-8a17-4420-ad7a-5d8be8944691 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:41.680989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa168d-863d-4c14-9b2a-b4287de77c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:41.681607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa168d-863d-4c14-9b2a-b4287de77c14 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:41.681988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa168d-863d-4c14-9b2a-b4287de77c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:41.682270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa168d-863d-4c14-9b2a-b4287de77c14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:41.682968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa168d-863d-4c14-9b2a-b4287de77c14 None admin] RBAC: Adding query filter params (interface=public, service_id=592ab5b06c424556bd2b4167b6264e55, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:41.683273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa168d-863d-4c14-9b2a-b4287de77c14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:41.684599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa168d-863d-4c14-9b2a-b4287de77c14 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:41.689427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 646/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Aug 11 09:44:41 2022] GET /identity/v3/endpoints?service_id=592ab5b06c424556bd2b4167b6264e55&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:41.695403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e9009-cfd9-412b-bd63-a8ffa67a1ac9 None admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:41.773789 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:41.790885 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46e9009-cfd9-412b-bd63-a8ffa67a1ac9 None admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:41.823915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef59f0-e766-447b-af10-c252c74a1c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:41.824505 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef59f0-e766-447b-af10-c252c74a1c8c None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:41.824786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef59f0-e766-447b-af10-c252c74a1c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:41.825079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef59f0-e766-447b-af10-c252c74a1c8c None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:41.825679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef59f0-e766-447b-af10-c252c74a1c8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:41.827027 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef59f0-e766-447b-af10-c252c74a1c8c None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:41.833518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 655/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Aug 11 09:44:41 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 09:44:43.850621 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f4a73-3c9d-43ec-9d83-6db3822e4576 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:43.851149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f4a73-3c9d-43ec-9d83-6db3822e4576 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:43.851149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f4a73-3c9d-43ec-9d83-6db3822e4576 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:43.852570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 647/1302] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:44:43.858286 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0572063-7de4-4e5e-8e00-23b4ced807f9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:43.858711 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0572063-7de4-4e5e-8e00-23b4ced807f9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:43.858858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0572063-7de4-4e5e-8e00-23b4ced807f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:43.876813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c0572063-7de4-4e5e-8e00-23b4ced807f9 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:44:43.878030 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c0572063-7de4-4e5e-8e00-23b4ced807f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:44:43.942932 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 656/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:44:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 09:44:43.967057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f4a73-3c9d-43ec-9d83-6db3822e4576 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:44.061968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:44.068448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f4a73-3c9d-43ec-9d83-6db3822e4576 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:44.102209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2863a-797f-4421-9655-7e6ad8325d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:44.102773 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2863a-797f-4421-9655-7e6ad8325d34 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:44.103054 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2863a-797f-4421-9655-7e6ad8325d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:44.103328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2863a-797f-4421-9655-7e6ad8325d34 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:44.103915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2863a-797f-4421-9655-7e6ad8325d34 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:44.105582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2863a-797f-4421-9655-7e6ad8325d34 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-dad2863a-797f-4421-9655-7e6ad8325d34 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 09:44:44.115040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 09:44:44.121413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 11 09:44:44.122275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 648/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Aug 11 09:44:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 09:44:44.123057 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0572063-7de4-4e5e-8e00-23b4ced807f9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:44.195870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:44.218220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0572063-7de4-4e5e-8e00-23b4ced807f9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:44.250660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37332d85-180d-4eef-b53e-030081ceff2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:44.251259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37332d85-180d-4eef-b53e-030081ceff2e None admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:44.251532 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37332d85-180d-4eef-b53e-030081ceff2e None admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:44.251801 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37332d85-180d-4eef-b53e-030081ceff2e None admin] PATH_INFO: `/v3/services` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:44.252423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37332d85-180d-4eef-b53e-030081ceff2e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:44:44.252723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37332d85-180d-4eef-b53e-030081ceff2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:44.254122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37332d85-180d-4eef-b53e-030081ceff2e None admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:44.258963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 657/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Aug 11 09:44:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:44:44.264767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2863a-797f-4421-9655-7e6ad8325d34 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:44:44.366642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2863a-797f-4421-9655-7e6ad8325d34 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:44:44.400059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f5ddb-f1f2-450b-b535-6590644128f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:44:44.400664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f5ddb-f1f2-450b-b535-6590644128f8 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:44:44.400943 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f5ddb-f1f2-450b-b535-6590644128f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:44:44.401245 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f5ddb-f1f2-450b-b535-6590644128f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:44:44.401911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f5ddb-f1f2-450b-b535-6590644128f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:44:44.403256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f5ddb-f1f2-450b-b535-6590644128f8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:44:44.477208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 649/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:44:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:44:54.266430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=30 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:44:54.472538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:37.706746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759e62f-aed0-453c-bf6e-95c19d23a7fb None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:37.706746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759e62f-aed0-453c-bf6e-95c19d23a7fb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:37.706746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759e62f-aed0-453c-bf6e-95c19d23a7fb None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:37.712536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fe162-ae48-472e-bdbc-c3798401ca95 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:37.712947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fe162-ae48-472e-bdbc-c3798401ca95 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:37.713107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fe162-ae48-472e-bdbc-c3798401ca95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:37.715937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 658/1307] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:51:37 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:51:37.837975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ec1fe162-ae48-472e-bdbc-c3798401ca95 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:51:37.839517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ec1fe162-ae48-472e-bdbc-c3798401ca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:37.992485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 650/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:51:38.048629 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f23af-9e27-48b8-94c3-f9059266dcb9 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:38.048849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f23af-9e27-48b8-94c3-f9059266dcb9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:38.049361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f23af-9e27-48b8-94c3-f9059266dcb9 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:38.050925 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 659/1309] 10.222.0.17 () {54 vars in 965 bytes} [Thu Aug 11 09:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:51:38.058505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8106e-3aa7-4420-a3f9-179352f8b4f0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:38.058824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8106e-3aa7-4420-a3f9-179352f8b4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:38.059104 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8106e-3aa7-4420-a3f9-179352f8b4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:38.141860 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fec8106e-3aa7-4420-a3f9-179352f8b4f0 None None] MFA Rules not processed for user `4fc6c3d1ce8645808c22d1b59b15a100`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:51:38.143119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fec8106e-3aa7-4420-a3f9-179352f8b4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:38.165757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:38.229907 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 651/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Aug 11 09:51:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:51:38.239068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f23af-9e27-48b8-94c3-f9059266dcb9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:51:38.240853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-679f23af-9e27-48b8-94c3-f9059266dcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:38.353526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f23af-9e27-48b8-94c3-f9059266dcb9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:51:38.437305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2e584-3068-40b0-91cf-27308e66e962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:51:38.437929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2e584-3068-40b0-91cf-27308e66e962 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:38.438210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2e584-3068-40b0-91cf-27308e66e962 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:38.438483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2e584-3068-40b0-91cf-27308e66e962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:38.440213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bdb2e584-3068-40b0-91cf-27308e66e962 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:38.505084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:38.563679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2e584-3068-40b0-91cf-27308e66e962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:51:38.565182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2e584-3068-40b0-91cf-27308e66e962 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:51:38.586675 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 660/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:51:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 09:51:40.891062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b2a8-1a9f-4264-a27b-34664406096d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:40.891599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b2a8-1a9f-4264-a27b-34664406096d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:40.891599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14b2a8-1a9f-4264-a27b-34664406096d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:40.892983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 652/1312] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:51:40.899081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9327b-0355-4e4a-b02a-aa30887a13bb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:40.899583 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9327b-0355-4e4a-b02a-aa30887a13bb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:40.899659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9327b-0355-4e4a-b02a-aa30887a13bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:40.978064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-74f9327b-0355-4e4a-b02a-aa30887a13bb None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:51:40.979288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-74f9327b-0355-4e4a-b02a-aa30887a13bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:41.093664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 661/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:51:41.146772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5234c-4329-421b-97bd-6f67e1e3ad67 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:41.146959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5234c-4329-421b-97bd-6f67e1e3ad67 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:41.147239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5234c-4329-421b-97bd-6f67e1e3ad67 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:41.148814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 653/1314] 10.222.0.17 () {54 vars in 965 bytes} [Thu Aug 11 09:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:51:41.155515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2099d-7d01-4130-a675-61b831ccb4e8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:41.155810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2099d-7d01-4130-a675-61b831ccb4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:41.156070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2099d-7d01-4130-a675-61b831ccb4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:41.169239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:41.246404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-07a2099d-7d01-4130-a675-61b831ccb4e8 None None] MFA Rules not processed for user `4fc6c3d1ce8645808c22d1b59b15a100`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:51:41.247615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-07a2099d-7d01-4130-a675-61b831ccb4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:41.322967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 662/1315] 10.222.0.17 () {58 vars in 1062 bytes} [Thu Aug 11 09:51:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:51:41.331776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5234c-4329-421b-97bd-6f67e1e3ad67 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:51:41.333647 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-69b5234c-4329-421b-97bd-6f67e1e3ad67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:41.436520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5234c-4329-421b-97bd-6f67e1e3ad67 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:51:41.509685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:41.512571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88caf2-a679-40bc-ac83-25b0128524d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:51:41.513223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88caf2-a679-40bc-ac83-25b0128524d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:41.513501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88caf2-a679-40bc-ac83-25b0128524d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:41.513770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88caf2-a679-40bc-ac83-25b0128524d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:41.515375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-af88caf2-a679-40bc-ac83-25b0128524d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:41.748412 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88caf2-a679-40bc-ac83-25b0128524d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:51:41.749803 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88caf2-a679-40bc-ac83-25b0128524d0 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:51:41.767511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 654/1316] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:51:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:51:44.564137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a59e3d-f36c-4296-8b53-efe1fbd6aea6 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:44.564685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a59e3d-f36c-4296-8b53-efe1fbd6aea6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:44.564685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a59e3d-f36c-4296-8b53-efe1fbd6aea6 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:44.566051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 663/1317] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:51:44.571980 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5471f-d1e9-4dc4-8156-ff0de9cf69c1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:44.572421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5471f-d1e9-4dc4-8156-ff0de9cf69c1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:44.572523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5471f-d1e9-4dc4-8156-ff0de9cf69c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:44.638296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d7a5471f-d1e9-4dc4-8156-ff0de9cf69c1 None None] MFA Rules not processed for user `0b773d1df61d4a3b933926aeff31873c`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:51:44.639504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d7a5471f-d1e9-4dc4-8156-ff0de9cf69c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:44.690674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:44.734662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 655/1318] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:51:44 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:51:44.775573 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a59e3d-f36c-4296-8b53-efe1fbd6aea6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:51:44.777171 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-51a59e3d-f36c-4296-8b53-efe1fbd6aea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:44.877332 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a59e3d-f36c-4296-8b53-efe1fbd6aea6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:51:44.880104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:44.950360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b369a3-ea37-4693-af86-8a9e2045a26b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:51:44.950959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b369a3-ea37-4693-af86-8a9e2045a26b service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:44.951231 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b369a3-ea37-4693-af86-8a9e2045a26b service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:44.951496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b369a3-ea37-4693-af86-8a9e2045a26b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:44.953111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-00b369a3-ea37-4693-af86-8a9e2045a26b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:45.057458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b369a3-ea37-4693-af86-8a9e2045a26b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:51:45.058883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b369a3-ea37-4693-af86-8a9e2045a26b service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:51:45.077584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 664/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:51:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:51:47.989288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e8494-270b-466e-8351-8d0e1b27fca4 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:47.989835 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e8494-270b-466e-8351-8d0e1b27fca4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:47.989932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e8494-270b-466e-8351-8d0e1b27fca4 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:47.991287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 656/1320] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:51:47.996296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5158f-bc25-4e87-bc23-097b82bd6861 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:47.996705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5158f-bc25-4e87-bc23-097b82bd6861 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:47.996845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5158f-bc25-4e87-bc23-097b82bd6861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:48.037755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:48.061958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-74a5158f-bc25-4e87-bc23-097b82bd6861 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:51:48.063180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-74a5158f-bc25-4e87-bc23-097b82bd6861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:48.158046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 665/1321] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:51:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:51:48.197587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e8494-270b-466e-8351-8d0e1b27fca4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:51:48.199126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e80e8494-270b-466e-8351-8d0e1b27fca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:48.299722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e8494-270b-466e-8351-8d0e1b27fca4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:51:48.372398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1263c-a9c6-4863-b3bf-3c97673c0313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:51:48.372990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1263c-a9c6-4863-b3bf-3c97673c0313 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:48.373296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1263c-a9c6-4863-b3bf-3c97673c0313 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:48.373563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1263c-a9c6-4863-b3bf-3c97673c0313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:48.375137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-25b1263c-a9c6-4863-b3bf-3c97673c0313 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:48.382018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:48.481389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1263c-a9c6-4863-b3bf-3c97673c0313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:51:48.482798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1263c-a9c6-4863-b3bf-3c97673c0313 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:51:48.501256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 657/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:51:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:51:51.616919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71297341-35a3-4f01-ba55-63be1d87b988 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:51.617520 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71297341-35a3-4f01-ba55-63be1d87b988 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:51.617520 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71297341-35a3-4f01-ba55-63be1d87b988 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:51.618863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 666/1323] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:51:51.624233 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376964d5-ab79-42ec-ad59-ceee0bfce2a1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:51.624643 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376964d5-ab79-42ec-ad59-ceee0bfce2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:51.624779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376964d5-ab79-42ec-ad59-ceee0bfce2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:51.690629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-376964d5-ab79-42ec-ad59-ceee0bfce2a1 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:51:51.691839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-376964d5-ab79-42ec-ad59-ceee0bfce2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:51.752258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:51.795212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 658/1324] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:51:51.834097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71297341-35a3-4f01-ba55-63be1d87b988 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:51:51.835737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-71297341-35a3-4f01-ba55-63be1d87b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:51.947395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71297341-35a3-4f01-ba55-63be1d87b988 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:51:51.960632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:52.024771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa23338-0287-456a-9a22-188dd62d048f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:51:52.025407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa23338-0287-456a-9a22-188dd62d048f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:52.025687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa23338-0287-456a-9a22-188dd62d048f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:52.025961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa23338-0287-456a-9a22-188dd62d048f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:52.027565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-baa23338-0287-456a-9a22-188dd62d048f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:52.137096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa23338-0287-456a-9a22-188dd62d048f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:51:52.138522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa23338-0287-456a-9a22-188dd62d048f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:51:52.156583 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 667/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:51:54.233514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaec51-41a8-4784-8145-2ff988748194 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:54.234066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaec51-41a8-4784-8145-2ff988748194 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:54.234066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaec51-41a8-4784-8145-2ff988748194 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:54.235404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 659/1326] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:51:54.241305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cfbbf-3bdf-439e-b4cf-de8a7ea9e4fb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:54.241729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cfbbf-3bdf-439e-b4cf-de8a7ea9e4fb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:54.241867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cfbbf-3bdf-439e-b4cf-de8a7ea9e4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:54.292594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:54.311321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-915cfbbf-3bdf-439e-b4cf-de8a7ea9e4fb None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:51:54.312653 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-915cfbbf-3bdf-439e-b4cf-de8a7ea9e4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:54.410967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 668/1327] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:51:54.449599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaec51-41a8-4784-8145-2ff988748194 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:51:54.451216 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4ecaec51-41a8-4784-8145-2ff988748194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:54.562306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaec51-41a8-4784-8145-2ff988748194 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:51:54.637112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce263dc-0356-4c34-a532-46d5af6c19fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:51:54.637720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce263dc-0356-4c34-a532-46d5af6c19fe service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:54.637994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce263dc-0356-4c34-a532-46d5af6c19fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:54.638286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce263dc-0356-4c34-a532-46d5af6c19fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:54.639894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8ce263dc-0356-4c34-a532-46d5af6c19fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:54.657565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:54.756952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce263dc-0356-4c34-a532-46d5af6c19fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:51:54.758443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce263dc-0356-4c34-a532-46d5af6c19fe service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:51:54.777238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 660/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:51:56.828867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a524e73-454d-405e-a182-35fef668e849 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:56.829432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a524e73-454d-405e-a182-35fef668e849 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:56.829432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a524e73-454d-405e-a182-35fef668e849 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:56.830781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 669/1329] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:51:56.836235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad0de5-75d7-44a0-81e8-75cf2580b5a2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:56.836638 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad0de5-75d7-44a0-81e8-75cf2580b5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:56.836797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad0de5-75d7-44a0-81e8-75cf2580b5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:56.903799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9dad0de5-75d7-44a0-81e8-75cf2580b5a2 None None] MFA Rules not processed for user `0b773d1df61d4a3b933926aeff31873c`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:51:56.905008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9dad0de5-75d7-44a0-81e8-75cf2580b5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:56.969705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:57.003064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 661/1330] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:51:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:51:57.040505 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a524e73-454d-405e-a182-35fef668e849 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:51:57.042257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5a524e73-454d-405e-a182-35fef668e849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:57.163001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a524e73-454d-405e-a182-35fef668e849 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:51:57.184769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:51:57.239380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05831b-a3fc-44e0-b8f4-2963633a1a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:51:57.240535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05831b-a3fc-44e0-b8f4-2963633a1a85 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:51:57.240958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05831b-a3fc-44e0-b8f4-2963633a1a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:51:57.241276 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05831b-a3fc-44e0-b8f4-2963633a1a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:51:57.242875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5d05831b-a3fc-44e0-b8f4-2963633a1a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:51:57.353991 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05831b-a3fc-44e0-b8f4-2963633a1a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:51:57.355426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05831b-a3fc-44e0-b8f4-2963633a1a85 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:51:57.374008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 670/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:51:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:00.071705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf3bbe-26ff-4f4b-9dd9-0f1a107b281c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:00.072276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf3bbe-26ff-4f4b-9dd9-0f1a107b281c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:00.072276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf3bbe-26ff-4f4b-9dd9-0f1a107b281c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:00.073702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 662/1332] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:00.078872 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7b85b-4a9d-4cf6-8b4d-178c71041aae None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:00.079268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7b85b-4a9d-4cf6-8b4d-178c71041aae None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:00.079412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7b85b-4a9d-4cf6-8b4d-178c71041aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:00.142068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:00.144635 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-cad7b85b-4a9d-4cf6-8b4d-178c71041aae None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:00.146383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cad7b85b-4a9d-4cf6-8b4d-178c71041aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:00.242722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 671/1333] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:00.278788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf3bbe-26ff-4f4b-9dd9-0f1a107b281c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:00.280320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-90bf3bbe-26ff-4f4b-9dd9-0f1a107b281c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:00.383435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf3bbe-26ff-4f4b-9dd9-0f1a107b281c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:00.457148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6acfc-f79a-493b-951b-91db8c4d9135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:00.463173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6acfc-f79a-493b-951b-91db8c4d9135 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:00.463173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6acfc-f79a-493b-951b-91db8c4d9135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:00.463173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6acfc-f79a-493b-951b-91db8c4d9135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:00.463173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-87c6acfc-f79a-493b-951b-91db8c4d9135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:00.488549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:00.569665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6acfc-f79a-493b-951b-91db8c4d9135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:00.571119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6acfc-f79a-493b-951b-91db8c4d9135 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:00.589916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 663/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:02.613681 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc13609-07d0-4a0c-b957-ab97d0b2d992 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:02.614242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc13609-07d0-4a0c-b957-ab97d0b2d992 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:02.614242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc13609-07d0-4a0c-b957-ab97d0b2d992 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:02.615604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 672/1335] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:02.620866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216bce7-90a0-400f-8f2a-e08ec31ee3db None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:02.621301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216bce7-90a0-400f-8f2a-e08ec31ee3db None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:02.621588 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216bce7-90a0-400f-8f2a-e08ec31ee3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:02.687785 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5216bce7-90a0-400f-8f2a-e08ec31ee3db None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:02.689041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5216bce7-90a0-400f-8f2a-e08ec31ee3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:02.765404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:02.788465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 664/1336] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:52:02.827977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc13609-07d0-4a0c-b957-ab97d0b2d992 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:02.829753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-acc13609-07d0-4a0c-b957-ab97d0b2d992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:02.933903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc13609-07d0-4a0c-b957-ab97d0b2d992 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:02.962766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:03.008665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24b792-ab01-4850-ac2f-7896ea9797ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:03.009385 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24b792-ab01-4850-ac2f-7896ea9797ea service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:03.009668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24b792-ab01-4850-ac2f-7896ea9797ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:03.009942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24b792-ab01-4850-ac2f-7896ea9797ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:03.011598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7a24b792-ab01-4850-ac2f-7896ea9797ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:03.124360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24b792-ab01-4850-ac2f-7896ea9797ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:03.125818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24b792-ab01-4850-ac2f-7896ea9797ea service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:03.144234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 673/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:05.895832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e3cf9-aeb0-4c22-8570-1cd3a9479082 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:05.896365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e3cf9-aeb0-4c22-8570-1cd3a9479082 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:05.896365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e3cf9-aeb0-4c22-8570-1cd3a9479082 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:05.897768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 665/1338] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:05.903454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49836935-31c4-4f2f-a243-1adff59acc56 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:05.903875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49836935-31c4-4f2f-a243-1adff59acc56 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:05.903990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49836935-31c4-4f2f-a243-1adff59acc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:05.969903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-49836935-31c4-4f2f-a243-1adff59acc56 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:05.971121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-49836935-31c4-4f2f-a243-1adff59acc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:05.979790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:06.073460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 674/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:06.109858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e3cf9-aeb0-4c22-8570-1cd3a9479082 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:06.111390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7d7e3cf9-aeb0-4c22-8570-1cd3a9479082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:06.214558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e3cf9-aeb0-4c22-8570-1cd3a9479082 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:06.285776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5828449-7dd6-4a65-83fb-e8aea068767e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:06.286390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5828449-7dd6-4a65-83fb-e8aea068767e service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:06.286772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5828449-7dd6-4a65-83fb-e8aea068767e service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:06.287053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5828449-7dd6-4a65-83fb-e8aea068767e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:06.288693 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e5828449-7dd6-4a65-83fb-e8aea068767e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:06.324456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:06.398238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5828449-7dd6-4a65-83fb-e8aea068767e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:06.399660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5828449-7dd6-4a65-83fb-e8aea068767e service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:06.418124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 666/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:10.750788 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687114c-5dca-4d3f-b74d-76d9f5d1c59a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:10.751337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687114c-5dca-4d3f-b74d-76d9f5d1c59a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:10.751337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687114c-5dca-4d3f-b74d-76d9f5d1c59a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:10.752821 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 675/1341] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:10.758927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b560df5-0259-43a1-9ae9-1daaff5431a4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:10.759328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b560df5-0259-43a1-9ae9-1daaff5431a4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:10.759452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b560df5-0259-43a1-9ae9-1daaff5431a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:10.827514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8b560df5-0259-43a1-9ae9-1daaff5431a4 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:10.828640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8b560df5-0259-43a1-9ae9-1daaff5431a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:10.916389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:10.928446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 667/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:10.969328 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687114c-5dca-4d3f-b74d-76d9f5d1c59a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:10.970871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e687114c-5dca-4d3f-b74d-76d9f5d1c59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:11.076993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687114c-5dca-4d3f-b74d-76d9f5d1c59a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:11.115771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:11.152781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae6840-8c78-4b7f-a959-5e68247b86cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:11.153422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6840-8c78-4b7f-a959-5e68247b86cd service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:11.153703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6840-8c78-4b7f-a959-5e68247b86cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:11.153977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae6840-8c78-4b7f-a959-5e68247b86cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:11.155580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f7ae6840-8c78-4b7f-a959-5e68247b86cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:11.270011 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae6840-8c78-4b7f-a959-5e68247b86cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:11.271543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae6840-8c78-4b7f-a959-5e68247b86cd service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:11.290039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 676/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:14.354303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4badc9-c82b-4306-8632-6058d0040643 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:14.354822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4badc9-c82b-4306-8632-6058d0040643 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:14.354822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4badc9-c82b-4306-8632-6058d0040643 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:14.356146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 668/1344] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:14.361685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182d4d1-0c41-453b-891b-60a21a2c37cb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:14.362120 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182d4d1-0c41-453b-891b-60a21a2c37cb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:14.362215 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182d4d1-0c41-453b-891b-60a21a2c37cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:14.430064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2182d4d1-0c41-453b-891b-60a21a2c37cb None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:14.431285 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2182d4d1-0c41-453b-891b-60a21a2c37cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:14.450081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:14.530561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 677/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:52:14.570185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4badc9-c82b-4306-8632-6058d0040643 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:14.571865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3f4badc9-c82b-4306-8632-6058d0040643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:14.676066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4badc9-c82b-4306-8632-6058d0040643 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:14.750208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b24fa-14a9-42de-bfa1-3abe7658a9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:14.750824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b24fa-14a9-42de-bfa1-3abe7658a9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:14.751104 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b24fa-14a9-42de-bfa1-3abe7658a9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:14.751383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b24fa-14a9-42de-bfa1-3abe7658a9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:14.753002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b84b24fa-14a9-42de-bfa1-3abe7658a9a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:14.800439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:14.869437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b24fa-14a9-42de-bfa1-3abe7658a9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:14.870872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b24fa-14a9-42de-bfa1-3abe7658a9a6 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:14.891413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 669/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:19.171866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484c873-cd6c-4fb9-acc1-3f147eb88971 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:19.172325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484c873-cd6c-4fb9-acc1-3f147eb88971 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:19.173294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484c873-cd6c-4fb9-acc1-3f147eb88971 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:19.174813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 678/1347] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:19.180645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9ae4b-1ea7-4185-812f-c34236344e97 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:19.181173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9ae4b-1ea7-4185-812f-c34236344e97 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:19.181255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9ae4b-1ea7-4185-812f-c34236344e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:19.266091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1df9ae4b-1ea7-4185-812f-c34236344e97 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:19.267478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1df9ae4b-1ea7-4185-812f-c34236344e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:19.372814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:19.376971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 670/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:52:19.415147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484c873-cd6c-4fb9-acc1-3f147eb88971 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:19.416864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f484c873-cd6c-4fb9-acc1-3f147eb88971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:19.573261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484c873-cd6c-4fb9-acc1-3f147eb88971 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:19.655313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:19.686165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06052ef9-ebcc-436f-9fee-0c17f29cb95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:19.686779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06052ef9-ebcc-436f-9fee-0c17f29cb95c service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:19.687066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06052ef9-ebcc-436f-9fee-0c17f29cb95c service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:19.687363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06052ef9-ebcc-436f-9fee-0c17f29cb95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:19.689212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-06052ef9-ebcc-436f-9fee-0c17f29cb95c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:19.830582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06052ef9-ebcc-436f-9fee-0c17f29cb95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:19.832802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06052ef9-ebcc-436f-9fee-0c17f29cb95c service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:19.864018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 679/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:24.800633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8bc9d-cff7-4da4-8b85-cc952a247687 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:24.801161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8bc9d-cff7-4da4-8b85-cc952a247687 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:24.801252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8bc9d-cff7-4da4-8b85-cc952a247687 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:24.802566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 671/1350] 10.222.0.17 () {54 vars in 921 bytes} [Thu Aug 11 09:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:24.807906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518741cf-7ca3-49bd-9ba2-3ef7d93464ba None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:24.814480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518741cf-7ca3-49bd-9ba2-3ef7d93464ba None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:24.814480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518741cf-7ca3-49bd-9ba2-3ef7d93464ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:24.913982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-518741cf-7ca3-49bd-9ba2-3ef7d93464ba None None] MFA Rules not processed for user `10ce125d46b545499de1fcf7072963e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:24.915427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-518741cf-7ca3-49bd-9ba2-3ef7d93464ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:24.939076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:25.100823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 680/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Aug 11 09:52:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:29.423814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20489562-ca1e-421e-a129-2b83a58635b3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:29.424337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20489562-ca1e-421e-a129-2b83a58635b3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:29.424337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20489562-ca1e-421e-a129-2b83a58635b3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:29.425905 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 672/1352] 10.222.0.17 () {54 vars in 921 bytes} [Thu Aug 11 09:52:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:29.430849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a8790-9ab4-4a02-ba33-0b9fa8f2f88d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:29.431741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a8790-9ab4-4a02-ba33-0b9fa8f2f88d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:29.432252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a8790-9ab4-4a02-ba33-0b9fa8f2f88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:29.519489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3b2a8790-9ab4-4a02-ba33-0b9fa8f2f88d None None] MFA Rules not processed for user `10ce125d46b545499de1fcf7072963e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:29.521342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3b2a8790-9ab4-4a02-ba33-0b9fa8f2f88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:29.605655 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:29.647763 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 681/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Aug 11 09:52:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:30.891834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2befe-1358-4331-8329-f7b7c071ad28 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:30.892360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2befe-1358-4331-8329-f7b7c071ad28 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:30.892451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2befe-1358-4331-8329-f7b7c071ad28 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:30.893829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 673/1354] 10.222.0.17 () {54 vars in 921 bytes} [Thu Aug 11 09:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:30.899593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15645bac-fcbc-4baf-8724-bb7ab94d7e7a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:30.900075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15645bac-fcbc-4baf-8724-bb7ab94d7e7a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:30.900193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15645bac-fcbc-4baf-8724-bb7ab94d7e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:30.977200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-15645bac-fcbc-4baf-8724-bb7ab94d7e7a None None] MFA Rules not processed for user `10ce125d46b545499de1fcf7072963e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:30.978406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-15645bac-fcbc-4baf-8724-bb7ab94d7e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:31.087856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 682/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Aug 11 09:52:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:34.491474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32ca14-c129-4762-ac59-feb79eb2e9b9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:34.491987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32ca14-c129-4762-ac59-feb79eb2e9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:34.491987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32ca14-c129-4762-ac59-feb79eb2e9b9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:34.498203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 674/1356] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:34.503167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daec966-dc77-4d94-92b3-7b5a0ccd2c3a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:34.503642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daec966-dc77-4d94-92b3-7b5a0ccd2c3a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:34.503754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daec966-dc77-4d94-92b3-7b5a0ccd2c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:34.555975 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:34.601334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5daec966-dc77-4d94-92b3-7b5a0ccd2c3a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:34.602822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5daec966-dc77-4d94-92b3-7b5a0ccd2c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:34.717186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 683/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:52:34.773131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b664440-4b4a-49db-b7ad-bb6c6692c024 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:34.773767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b664440-4b4a-49db-b7ad-bb6c6692c024 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:34.774272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b664440-4b4a-49db-b7ad-bb6c6692c024 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:34.776193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 675/1358] 10.222.0.17 () {54 vars in 962 bytes} [Thu Aug 11 09:52:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:34.784216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b962d5-d24a-4e80-af39-e64f331b8057 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:34.784500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b962d5-d24a-4e80-af39-e64f331b8057 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:34.788294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b962d5-d24a-4e80-af39-e64f331b8057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:34.870040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-99b962d5-d24a-4e80-af39-e64f331b8057 None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:34.871290 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-99b962d5-d24a-4e80-af39-e64f331b8057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:34.923766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:34.993529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 684/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Aug 11 09:52:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:35.001225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b664440-4b4a-49db-b7ad-bb6c6692c024 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:35.002955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0b664440-4b4a-49db-b7ad-bb6c6692c024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:35.156183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b664440-4b4a-49db-b7ad-bb6c6692c024 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:35.243071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:35.283630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26ae87-dc20-4afd-8510-cf7e75c1e436 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:35.284243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26ae87-dc20-4afd-8510-cf7e75c1e436 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:35.284541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26ae87-dc20-4afd-8510-cf7e75c1e436 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:35.284806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26ae87-dc20-4afd-8510-cf7e75c1e436 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:35.286509 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cf26ae87-dc20-4afd-8510-cf7e75c1e436 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:35.392560 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26ae87-dc20-4afd-8510-cf7e75c1e436 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:35.394486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26ae87-dc20-4afd-8510-cf7e75c1e436 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:35.414563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 676/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:37.640980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332a2b2-0ea0-4fa4-abff-c8f7d9f93994 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:37.641552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332a2b2-0ea0-4fa4-abff-c8f7d9f93994 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:37.641552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332a2b2-0ea0-4fa4-abff-c8f7d9f93994 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:37.642952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 685/1361] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:37.647910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e55e-046c-4e2b-b64a-fb4f7f537837 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:37.648310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e55e-046c-4e2b-b64a-fb4f7f537837 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:37.648477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75e55e-046c-4e2b-b64a-fb4f7f537837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:37.726591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3b75e55e-046c-4e2b-b64a-fb4f7f537837 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:37.728093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3b75e55e-046c-4e2b-b64a-fb4f7f537837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:37.769591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:37.834481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 677/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:37.881466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332a2b2-0ea0-4fa4-abff-c8f7d9f93994 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:37.883110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c332a2b2-0ea0-4fa4-abff-c8f7d9f93994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:38.063979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332a2b2-0ea0-4fa4-abff-c8f7d9f93994 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:38.078168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:38.195959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d81f5d-00f8-4d2d-be9b-025228faced5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:38.196985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d81f5d-00f8-4d2d-be9b-025228faced5 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:38.197391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d81f5d-00f8-4d2d-be9b-025228faced5 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:38.197757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d81f5d-00f8-4d2d-be9b-025228faced5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:38.199535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-58d81f5d-00f8-4d2d-be9b-025228faced5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:38.335703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d81f5d-00f8-4d2d-be9b-025228faced5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:38.337500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d81f5d-00f8-4d2d-be9b-025228faced5 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:38.358196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 686/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:40.859313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbb794-98de-432b-9b6d-7a7db1be51c1 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:40.860308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbb794-98de-432b-9b6d-7a7db1be51c1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:40.860740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbb794-98de-432b-9b6d-7a7db1be51c1 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:40.862225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 678/1364] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:40.868034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf949d4-c946-4b27-9cb1-b43f3dd9de4c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:40.868522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf949d4-c946-4b27-9cb1-b43f3dd9de4c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:40.868624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf949d4-c946-4b27-9cb1-b43f3dd9de4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:40.887597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:40.945412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3cf949d4-c946-4b27-9cb1-b43f3dd9de4c None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:40.946678 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3cf949d4-c946-4b27-9cb1-b43f3dd9de4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:41.054074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 687/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:52:41.100158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fe55d-f9ce-47ff-9d9a-15cb30d44825 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:41.100522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fe55d-f9ce-47ff-9d9a-15cb30d44825 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:41.100801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fe55d-f9ce-47ff-9d9a-15cb30d44825 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:41.102290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 679/1366] 10.222.0.17 () {54 vars in 962 bytes} [Thu Aug 11 09:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:41.108506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268191d-6bd1-45b8-b21f-19e64f3c507d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:41.108914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268191d-6bd1-45b8-b21f-19e64f3c507d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:41.109414 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268191d-6bd1-45b8-b21f-19e64f3c507d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:41.196737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4268191d-6bd1-45b8-b21f-19e64f3c507d None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:41.198282 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4268191d-6bd1-45b8-b21f-19e64f3c507d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:41.239813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:41.325586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 688/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Aug 11 09:52:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:41.334783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fe55d-f9ce-47ff-9d9a-15cb30d44825 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:41.336381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5c6fe55d-f9ce-47ff-9d9a-15cb30d44825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:41.491816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fe55d-f9ce-47ff-9d9a-15cb30d44825 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:41.512375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:41.614508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b6acd-e2a1-41dc-a984-c74e1f996cac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:41.615118 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6acd-e2a1-41dc-a984-c74e1f996cac service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:41.615500 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6acd-e2a1-41dc-a984-c74e1f996cac service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:41.615776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6acd-e2a1-41dc-a984-c74e1f996cac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:41.617409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f31b6acd-e2a1-41dc-a984-c74e1f996cac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:41.712172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b6acd-e2a1-41dc-a984-c74e1f996cac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:41.713806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b6acd-e2a1-41dc-a984-c74e1f996cac service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:41.733323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 680/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 09:52:42.806056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3ecf5-46bb-45d1-9163-3feeb6893f2f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:42.806608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3ecf5-46bb-45d1-9163-3feeb6893f2f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:42.806608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3ecf5-46bb-45d1-9163-3feeb6893f2f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:42.808052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 689/1369] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:42.813200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfea0d-0ec9-45bb-8ea7-437fc6226c8c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:42.813633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfea0d-0ec9-45bb-8ea7-437fc6226c8c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:42.813633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfea0d-0ec9-45bb-8ea7-437fc6226c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:42.847764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:42.886249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3fbfea0d-0ec9-45bb-8ea7-437fc6226c8c None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:42.888006 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3fbfea0d-0ec9-45bb-8ea7-437fc6226c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:43.004691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 681/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:52:43.344008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f494c-d7a7-4f04-bf12-71be853c21e9 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:43.344289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f494c-d7a7-4f04-bf12-71be853c21e9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:43.344560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f494c-d7a7-4f04-bf12-71be853c21e9 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:43.346041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 690/1371] 10.222.0.17 () {54 vars in 964 bytes} [Thu Aug 11 09:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:43.351302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72331019-518d-47dd-af72-0fdaa459bc65 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:43.351584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72331019-518d-47dd-af72-0fdaa459bc65 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:43.351855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72331019-518d-47dd-af72-0fdaa459bc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:43.432738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-72331019-518d-47dd-af72-0fdaa459bc65 None None] MFA Rules not processed for user `029bb992f9974eec83c0fe7f9fa58826`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:43.434024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-72331019-518d-47dd-af72-0fdaa459bc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:43.483302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:43.515489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 682/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Thu Aug 11 09:52:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:43.523163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f494c-d7a7-4f04-bf12-71be853c21e9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:43.524742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-831f494c-d7a7-4f04-bf12-71be853c21e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:43.637154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f494c-d7a7-4f04-bf12-71be853c21e9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:43.679589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:43.715096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c78a0-77ca-4fda-87d1-05dfeab7c00d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:43.715710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c78a0-77ca-4fda-87d1-05dfeab7c00d service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:43.715992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c78a0-77ca-4fda-87d1-05dfeab7c00d service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:43.716265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c78a0-77ca-4fda-87d1-05dfeab7c00d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:43.717902 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-139c78a0-77ca-4fda-87d1-05dfeab7c00d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:43.833168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c78a0-77ca-4fda-87d1-05dfeab7c00d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:43.834673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c78a0-77ca-4fda-87d1-05dfeab7c00d service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:43.855794 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 691/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:44.005802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07abd38-78ec-494e-abb7-021842164b1d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:44.006355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07abd38-78ec-494e-abb7-021842164b1d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:44.006355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07abd38-78ec-494e-abb7-021842164b1d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:44.007684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 683/1374] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:44.013001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46063d44-3482-4fa4-867b-932eb7903fe8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:44.013404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46063d44-3482-4fa4-867b-932eb7903fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:44.013801 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46063d44-3482-4fa4-867b-932eb7903fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:44.086437 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-46063d44-3482-4fa4-867b-932eb7903fe8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:44.087683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-46063d44-3482-4fa4-867b-932eb7903fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:44.109342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:44.193339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 692/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:44.241756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07abd38-78ec-494e-abb7-021842164b1d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:44.243370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c07abd38-78ec-494e-abb7-021842164b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:44.406124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07abd38-78ec-494e-abb7-021842164b1d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:44.457754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:44.523186 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a33bc5-99a8-46d9-a0c4-45b3de143d95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:44.523862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a33bc5-99a8-46d9-a0c4-45b3de143d95 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:44.524677 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a33bc5-99a8-46d9-a0c4-45b3de143d95 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:44.524953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a33bc5-99a8-46d9-a0c4-45b3de143d95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:44.526760 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-49a33bc5-99a8-46d9-a0c4-45b3de143d95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:44.620774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a33bc5-99a8-46d9-a0c4-45b3de143d95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:44.622309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a33bc5-99a8-46d9-a0c4-45b3de143d95 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:44.640594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 684/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:46.900244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11226890-9e2d-4998-aa8b-09580c804c8a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:46.900791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11226890-9e2d-4998-aa8b-09580c804c8a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:46.900890 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11226890-9e2d-4998-aa8b-09580c804c8a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:46.902372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 693/1377] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:46.910522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d109e-8212-4871-bddf-a1c6f0199318 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:46.910522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d109e-8212-4871-bddf-a1c6f0199318 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:46.910522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d109e-8212-4871-bddf-a1c6f0199318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:46.978528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fc3d109e-8212-4871-bddf-a1c6f0199318 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:46.979805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fc3d109e-8212-4871-bddf-a1c6f0199318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:46.986514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:47.095151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 685/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:47.142828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11226890-9e2d-4998-aa8b-09580c804c8a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:47.144403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-11226890-9e2d-4998-aa8b-09580c804c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:47.296609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11226890-9e2d-4998-aa8b-09580c804c8a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:47.300198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:47.411319 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369f5e2-9e82-4191-9eb7-134aa667ab44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:47.411939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369f5e2-9e82-4191-9eb7-134aa667ab44 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:47.412211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369f5e2-9e82-4191-9eb7-134aa667ab44 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:47.412484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369f5e2-9e82-4191-9eb7-134aa667ab44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:47.414725 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a369f5e2-9e82-4191-9eb7-134aa667ab44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:47.524002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369f5e2-9e82-4191-9eb7-134aa667ab44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:47.526669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369f5e2-9e82-4191-9eb7-134aa667ab44 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:47.559055 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 694/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:49.746629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5201c61-eab3-4976-a422-0fa04a07cca8 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:49.747196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5201c61-eab3-4976-a422-0fa04a07cca8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:49.747196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5201c61-eab3-4976-a422-0fa04a07cca8 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:49.748651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 686/1380] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:49.753785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4d6bc-e87c-49fe-8954-9a0f2aea2b70 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:49.754169 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4d6bc-e87c-49fe-8954-9a0f2aea2b70 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:49.754349 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4d6bc-e87c-49fe-8954-9a0f2aea2b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:49.766139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:49.822970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-57c4d6bc-e87c-49fe-8954-9a0f2aea2b70 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:49.824170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-57c4d6bc-e87c-49fe-8954-9a0f2aea2b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:49.940358 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 695/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:49.980359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5201c61-eab3-4976-a422-0fa04a07cca8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:49.982332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d5201c61-eab3-4976-a422-0fa04a07cca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:50.130673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:50.142105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5201c61-eab3-4976-a422-0fa04a07cca8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:50.277152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85101f4c-d19d-4ba4-902a-a8d03388e432 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:50.277804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85101f4c-d19d-4ba4-902a-a8d03388e432 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:50.278078 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85101f4c-d19d-4ba4-902a-a8d03388e432 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:50.278353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85101f4c-d19d-4ba4-902a-a8d03388e432 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:50.280159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-85101f4c-d19d-4ba4-902a-a8d03388e432 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:50.386514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85101f4c-d19d-4ba4-902a-a8d03388e432 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:50.388184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85101f4c-d19d-4ba4-902a-a8d03388e432 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:50.400903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:50.411315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 687/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:52.633637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f31d16-0d1e-45fc-a7d9-f387c228f940 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:52.634267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f31d16-0d1e-45fc-a7d9-f387c228f940 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:52.634447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f31d16-0d1e-45fc-a7d9-f387c228f940 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:52.636218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 696/1383] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:52.642731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531b4d-977f-43ad-b55b-534a23f432e9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:52.643331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531b4d-977f-43ad-b55b-534a23f432e9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:52.643331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531b4d-977f-43ad-b55b-534a23f432e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:52.710756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ee531b4d-977f-43ad-b55b-534a23f432e9 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:52.711928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ee531b4d-977f-43ad-b55b-534a23f432e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:52.808378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 688/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:52.852316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f31d16-0d1e-45fc-a7d9-f387c228f940 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:52.854020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-29f31d16-0d1e-45fc-a7d9-f387c228f940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:52.946632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:53.027572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f31d16-0d1e-45fc-a7d9-f387c228f940 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:53.166040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cda85-ce8f-465b-8b8e-c20ff578d6c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:53.166724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cda85-ce8f-465b-8b8e-c20ff578d6c6 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:53.167010 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cda85-ce8f-465b-8b8e-c20ff578d6c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:53.167320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cda85-ce8f-465b-8b8e-c20ff578d6c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:53.168976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ae4cda85-ce8f-465b-8b8e-c20ff578d6c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:53.223555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:53.267097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cda85-ce8f-465b-8b8e-c20ff578d6c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:53.268710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cda85-ce8f-465b-8b8e-c20ff578d6c6 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:53.286742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 697/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 09:52:56.298664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee31b3-661e-40ce-9864-9723f191e1b2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:56.299194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee31b3-661e-40ce-9864-9723f191e1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:56.299285 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee31b3-661e-40ce-9864-9723f191e1b2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:56.300660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 689/1386] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:56.307219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a17294-f980-4cd3-b6bd-fc9b25f4ad60 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:56.307742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a17294-f980-4cd3-b6bd-fc9b25f4ad60 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:56.308151 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a17294-f980-4cd3-b6bd-fc9b25f4ad60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:56.382092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-93a17294-f980-4cd3-b6bd-fc9b25f4ad60 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:56.383318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-93a17294-f980-4cd3-b6bd-fc9b25f4ad60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:56.481429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 698/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:56.530972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee31b3-661e-40ce-9864-9723f191e1b2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:56.537995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ddee31b3-661e-40ce-9864-9723f191e1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:56.626676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:56.709103 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee31b3-661e-40ce-9864-9723f191e1b2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:56.833280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd5f10-7070-4db7-b26c-01f5a93200e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:56.834126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd5f10-7070-4db7-b26c-01f5a93200e9 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:56.834505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd5f10-7070-4db7-b26c-01f5a93200e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:56.834844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd5f10-7070-4db7-b26c-01f5a93200e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:56.837110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fafd5f10-7070-4db7-b26c-01f5a93200e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:56.879783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:56.941261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd5f10-7070-4db7-b26c-01f5a93200e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:56.943397 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd5f10-7070-4db7-b26c-01f5a93200e9 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:56.968612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 690/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:52:59.125294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7d0b4-f879-4ce2-a61f-db3e8096073e None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:59.125896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7d0b4-f879-4ce2-a61f-db3e8096073e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:59.125896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7d0b4-f879-4ce2-a61f-db3e8096073e None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:59.127207 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 699/1389] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:59.133109 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55c16c-a1ca-4b50-97ce-2db615e1280d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:59.133532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55c16c-a1ca-4b50-97ce-2db615e1280d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:59.133654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55c16c-a1ca-4b50-97ce-2db615e1280d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:59.205697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8e55c16c-a1ca-4b50-97ce-2db615e1280d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:59.206943 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8e55c16c-a1ca-4b50-97ce-2db615e1280d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:59.234429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993682ef-5895-4766-ac2e-be1f4be5f21d None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:59.234714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993682ef-5895-4766-ac2e-be1f4be5f21d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:59.234992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993682ef-5895-4766-ac2e-be1f4be5f21d None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:59.236622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 700/1390] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:52:59.242273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0f40-094c-4185-9360-56d8621e80d6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:59.242553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0f40-094c-4185-9360-56d8621e80d6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:59.242928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0f40-094c-4185-9360-56d8621e80d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:59.278891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:59.306056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 691/1391] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:52:59.310622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-795f0f40-094c-4185-9360-56d8621e80d6 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:59.312467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-795f0f40-094c-4185-9360-56d8621e80d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:59.350390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55c16c-a1ca-4b50-97ce-2db615e1280d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:59.351637 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8e55c16c-a1ca-4b50-97ce-2db615e1280d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:59.365271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:59.410079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 701/1392] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:52:59.466537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff32d4d-6c9a-4987-ab20-73f2d1bc491c None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:59.466915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff32d4d-6c9a-4987-ab20-73f2d1bc491c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:59.467254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff32d4d-6c9a-4987-ab20-73f2d1bc491c None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:59.468644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 702/1393] 10.222.0.17 () {54 vars in 967 bytes} [Thu Aug 11 09:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:52:59.472083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55c16c-a1ca-4b50-97ce-2db615e1280d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:59.473334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0980243-08a1-4dfe-be9a-35b3b1d13784 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:59.473618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0980243-08a1-4dfe-be9a-35b3b1d13784 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:59.473887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0980243-08a1-4dfe-be9a-35b3b1d13784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:59.550909 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b0980243-08a1-4dfe-be9a-35b3b1d13784 None None] MFA Rules not processed for user `d83448e4c6704f7c8771132bd67eaa1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:52:59.552402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b0980243-08a1-4dfe-be9a-35b3b1d13784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:59.590742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efed578-25d1-4d68-aa8b-04f0abb49ee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:59.591388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efed578-25d1-4d68-aa8b-04f0abb49ee0 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:59.591696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efed578-25d1-4d68-aa8b-04f0abb49ee0 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:59.591968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efed578-25d1-4d68-aa8b-04f0abb49ee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:59.593691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5efed578-25d1-4d68-aa8b-04f0abb49ee0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:59.605327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:59.606695 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:59.641840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 703/1394] 10.222.0.17 () {58 vars in 1064 bytes} [Thu Aug 11 09:52:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:52:59.653468 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0980243-08a1-4dfe-be9a-35b3b1d13784 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:52:59.663053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0980243-08a1-4dfe-be9a-35b3b1d13784 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:52:59.694714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efed578-25d1-4d68-aa8b-04f0abb49ee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:59.696512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efed578-25d1-4d68-aa8b-04f0abb49ee0 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:59.716537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 692/1395] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 09:52:59.752463 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aae969-89b3-449b-aa7e-ef022ae41dcc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83448e4c6704f7c8771132bd67eaa1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:52:59.753124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aae969-89b3-449b-aa7e-ef022ae41dcc service glance] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:52:59.753408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aae969-89b3-449b-aa7e-ef022ae41dcc service glance] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:52:59.753677 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aae969-89b3-449b-aa7e-ef022ae41dcc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:52:59.755340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a7aae969-89b3-449b-aa7e-ef022ae41dcc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:52:59.867750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aae969-89b3-449b-aa7e-ef022ae41dcc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:52:59.869208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aae969-89b3-449b-aa7e-ef022ae41dcc service glance] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:52:59.883297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:52:59.887777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 704/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 09:53:00.791863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75998667-cf65-4531-9676-275079613108 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:00.792356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75998667-cf65-4531-9676-275079613108 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:00.792470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75998667-cf65-4531-9676-275079613108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:00.862738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-75998667-cf65-4531-9676-275079613108 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:00.863962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-75998667-cf65-4531-9676-275079613108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:00.951589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:00.954882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 693/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 09:53:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:01.002727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd2e13-974d-4e80-86a1-89fddc25a000 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:01.003194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd2e13-974d-4e80-86a1-89fddc25a000 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:01.003308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd2e13-974d-4e80-86a1-89fddc25a000 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:01.004839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 705/1398] 10.222.0.17 () {54 vars in 943 bytes} [Thu Aug 11 09:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:01.011386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126680-9f16-4ed6-8481-d7c992773274 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:01.011669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126680-9f16-4ed6-8481-d7c992773274 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:01.011947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8126680-9f16-4ed6-8481-d7c992773274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:01.089526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b8126680-9f16-4ed6-8481-d7c992773274 None None] MFA Rules not processed for user `915ee44218064af896020d008ff31957`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:01.090643 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b8126680-9f16-4ed6-8481-d7c992773274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:01.197751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 694/1399] 10.222.0.17 () {58 vars in 1040 bytes} [Thu Aug 11 09:53:01 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:01.207368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd2e13-974d-4e80-86a1-89fddc25a000 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:01.209287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-97dd2e13-974d-4e80-86a1-89fddc25a000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:01.352688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd2e13-974d-4e80-86a1-89fddc25a000 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:01.436692 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:01.455578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81176fa7-e991-4345-984f-87de4a193316 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:01.456178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81176fa7-e991-4345-984f-87de4a193316 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:01.456455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81176fa7-e991-4345-984f-87de4a193316 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:01.456726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81176fa7-e991-4345-984f-87de4a193316 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:01.458615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-81176fa7-e991-4345-984f-87de4a193316 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:01.539687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81176fa7-e991-4345-984f-87de4a193316 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:01.541291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81176fa7-e991-4345-984f-87de4a193316 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:01.549245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 706/1400] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 09:53:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 09:53:02.132789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991e5c9-f06c-435e-a43f-f4ab72847182 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:02.133437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991e5c9-f06c-435e-a43f-f4ab72847182 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:02.133437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991e5c9-f06c-435e-a43f-f4ab72847182 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:02.134706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 695/1401] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:02.139880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347322ae-0c0c-41df-a0f3-cc6a1260d521 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:02.140340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347322ae-0c0c-41df-a0f3-cc6a1260d521 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:02.140442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347322ae-0c0c-41df-a0f3-cc6a1260d521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:02.219018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-347322ae-0c0c-41df-a0f3-cc6a1260d521 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:02.220346 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-347322ae-0c0c-41df-a0f3-cc6a1260d521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:02.300227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:02.330344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 707/1402] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:02.376574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991e5c9-f06c-435e-a43f-f4ab72847182 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:02.378149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5991e5c9-f06c-435e-a43f-f4ab72847182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:02.446526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:02.547396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991e5c9-f06c-435e-a43f-f4ab72847182 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:02.675562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369a952-3d5d-4054-9ab5-9840a335a477 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:02.676302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369a952-3d5d-4054-9ab5-9840a335a477 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:02.676585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369a952-3d5d-4054-9ab5-9840a335a477 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:02.676859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369a952-3d5d-4054-9ab5-9840a335a477 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:02.680493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f369a952-3d5d-4054-9ab5-9840a335a477 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:02.701235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:02.778687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369a952-3d5d-4054-9ab5-9840a335a477 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:02.780314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369a952-3d5d-4054-9ab5-9840a335a477 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:02.800755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 696/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:04.996238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd964cb-d9c4-4318-a079-b7e4c49f959e None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:04.997287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd964cb-d9c4-4318-a079-b7e4c49f959e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:04.997579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd964cb-d9c4-4318-a079-b7e4c49f959e None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:04.999122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 708/1404] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:53:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:05.004176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1047-7320-4a2b-8711-352f74ede211 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:05.004587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1047-7320-4a2b-8711-352f74ede211 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:05.004857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd1047-7320-4a2b-8711-352f74ede211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:05.105650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bafd1047-7320-4a2b-8711-352f74ede211 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:05.106932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bafd1047-7320-4a2b-8711-352f74ede211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:05.205951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:05.213365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 697/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:05.256996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd964cb-d9c4-4318-a079-b7e4c49f959e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:05.258670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ddd964cb-d9c4-4318-a079-b7e4c49f959e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:05.449300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd964cb-d9c4-4318-a079-b7e4c49f959e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:05.515827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:05.580275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070cd80-65e0-4b86-bda1-1871eb3ac9ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:05.580891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070cd80-65e0-4b86-bda1-1871eb3ac9ab service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:05.581201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070cd80-65e0-4b86-bda1-1871eb3ac9ab service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:05.581477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070cd80-65e0-4b86-bda1-1871eb3ac9ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:05.583364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3070cd80-65e0-4b86-bda1-1871eb3ac9ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:05.679465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070cd80-65e0-4b86-bda1-1871eb3ac9ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:05.681110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070cd80-65e0-4b86-bda1-1871eb3ac9ab service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:05.699908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 709/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:07.991554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adcde1-b215-42b1-b92a-4702a950895b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:07.992111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adcde1-b215-42b1-b92a-4702a950895b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:07.992111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adcde1-b215-42b1-b92a-4702a950895b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:07.994249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 698/1407] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:08.000307 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64064b6d-b3ee-473a-822e-48d134a9f140 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:08.000815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64064b6d-b3ee-473a-822e-48d134a9f140 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:08.000935 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64064b6d-b3ee-473a-822e-48d134a9f140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:08.071552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-64064b6d-b3ee-473a-822e-48d134a9f140 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:08.072750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-64064b6d-b3ee-473a-822e-48d134a9f140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:08.086108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:08.184423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 710/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:53:08.226811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adcde1-b215-42b1-b92a-4702a950895b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:08.228337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-52adcde1-b215-42b1-b92a-4702a950895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:08.387102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adcde1-b215-42b1-b92a-4702a950895b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:08.461350 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:08.511674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd6f84-0d4e-4433-a74a-8c719117b29f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:08.512202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd6f84-0d4e-4433-a74a-8c719117b29f service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:08.512478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd6f84-0d4e-4433-a74a-8c719117b29f service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:08.512748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd6f84-0d4e-4433-a74a-8c719117b29f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:08.514436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b1dd6f84-0d4e-4433-a74a-8c719117b29f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:08.615304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd6f84-0d4e-4433-a74a-8c719117b29f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:08.616908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd6f84-0d4e-4433-a74a-8c719117b29f service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:08.636782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 699/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:18.180593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=38 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:18.634564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=64 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:25.445618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dedbc-ff1a-418b-af39-52a321a223da None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:25.446130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dedbc-ff1a-418b-af39-52a321a223da None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:25.446130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dedbc-ff1a-418b-af39-52a321a223da None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:25.447484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 711/1410] 10.222.0.17 () {54 vars in 918 bytes} [Thu Aug 11 09:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:25.452704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8eb638-f95e-493a-ac6f-1797cfc55c84 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:25.453151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8eb638-f95e-493a-ac6f-1797cfc55c84 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:25.453284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8eb638-f95e-493a-ac6f-1797cfc55c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:25.528228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9c8eb638-f95e-493a-ac6f-1797cfc55c84 None None] MFA Rules not processed for user `10ce125d46b545499de1fcf7072963e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:25.529467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9c8eb638-f95e-493a-ac6f-1797cfc55c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:25.633300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 700/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Aug 11 09:53:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:25.648158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631384a-24e9-46c8-a22c-3fc4c9e19802 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:25.648447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631384a-24e9-46c8-a22c-3fc4c9e19802 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:25.648723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631384a-24e9-46c8-a22c-3fc4c9e19802 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:25.650069 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 712/1412] 10.222.0.17 () {54 vars in 918 bytes} [Thu Aug 11 09:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:25.655112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202f6a1-8963-4d2f-8a80-70894be45e72 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:25.655401 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202f6a1-8963-4d2f-8a80-70894be45e72 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:25.655702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202f6a1-8963-4d2f-8a80-70894be45e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:25.728581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:25.731099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a202f6a1-8963-4d2f-8a80-70894be45e72 None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:25.732317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a202f6a1-8963-4d2f-8a80-70894be45e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:25.847557 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 701/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Aug 11 09:53:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:28.418798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49649514-7212-4e65-bdb0-d880cbd5f2c2 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:28.419334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49649514-7212-4e65-bdb0-d880cbd5f2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:28.419334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49649514-7212-4e65-bdb0-d880cbd5f2c2 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:28.420747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:28.426005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee11b9a-d1f6-4707-b443-ff428e990c67 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:28.426446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee11b9a-d1f6-4707-b443-ff428e990c67 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:28.426549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee11b9a-d1f6-4707-b443-ff428e990c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:28.493065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7ee11b9a-d1f6-4707-b443-ff428e990c67 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:28.494286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7ee11b9a-d1f6-4707-b443-ff428e990c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:28.563086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:28.611168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:28.654871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49649514-7212-4e65-bdb0-d880cbd5f2c2 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:28.656459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-49649514-7212-4e65-bdb0-d880cbd5f2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:28.801569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49649514-7212-4e65-bdb0-d880cbd5f2c2 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:28.873591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:28.912572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb76b8f-e901-489a-8daa-0c392e9e8241 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:28.913196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb76b8f-e901-489a-8daa-0c392e9e8241 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:28.913472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb76b8f-e901-489a-8daa-0c392e9e8241 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:28.913736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb76b8f-e901-489a-8daa-0c392e9e8241 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:28.915347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9bb76b8f-e901-489a-8daa-0c392e9e8241 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:29.004042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb76b8f-e901-489a-8daa-0c392e9e8241 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:29.006043 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb76b8f-e901-489a-8daa-0c392e9e8241 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:29.024201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 714/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:29.922083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51043343-dbb3-4040-b242-28d540b47fda None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:29.922653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51043343-dbb3-4040-b242-28d540b47fda None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:29.922653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51043343-dbb3-4040-b242-28d540b47fda None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:29.923958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:29.929445 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e8d2b-0a75-47e5-bbdb-4a6f6fc88f36 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:29.929858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e8d2b-0a75-47e5-bbdb-4a6f6fc88f36 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:29.929978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e8d2b-0a75-47e5-bbdb-4a6f6fc88f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:30.005809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ab6e8d2b-0a75-47e5-bbdb-4a6f6fc88f36 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:30.007022 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ab6e8d2b-0a75-47e5-bbdb-4a6f6fc88f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:30.033216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:30.101567 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:30.258203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51043343-dbb3-4040-b242-28d540b47fda None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:30.259750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-51043343-dbb3-4040-b242-28d540b47fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:30.404028 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51043343-dbb3-4040-b242-28d540b47fda None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:30.443092 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:30.515685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d178e3d-253d-4d96-86b2-3f37ddc5fcf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:30.516286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d178e3d-253d-4d96-86b2-3f37ddc5fcf6 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:30.516560 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d178e3d-253d-4d96-86b2-3f37ddc5fcf6 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:30.516830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d178e3d-253d-4d96-86b2-3f37ddc5fcf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:30.518449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9d178e3d-253d-4d96-86b2-3f37ddc5fcf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:30.607763 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d178e3d-253d-4d96-86b2-3f37ddc5fcf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:30.609385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d178e3d-253d-4d96-86b2-3f37ddc5fcf6 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:30.627183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 704/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 09:53:31.526359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf9e66-e5ab-4bc2-8701-06a0f8397e2f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:31.526953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf9e66-e5ab-4bc2-8701-06a0f8397e2f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:31.526953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf9e66-e5ab-4bc2-8701-06a0f8397e2f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:31.528238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:31.533308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283f6d1-177e-4d77-a895-cc9b15746882 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:31.533745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283f6d1-177e-4d77-a895-cc9b15746882 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:31.533857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283f6d1-177e-4d77-a895-cc9b15746882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:31.588051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:31.609538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e283f6d1-177e-4d77-a895-cc9b15746882 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:31.610750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e283f6d1-177e-4d77-a895-cc9b15746882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:31.707119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:31.860660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf9e66-e5ab-4bc2-8701-06a0f8397e2f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:31.862160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-88bf9e66-e5ab-4bc2-8701-06a0f8397e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:32.008999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf9e66-e5ab-4bc2-8701-06a0f8397e2f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:32.021060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:32.122833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f275d-7d60-42d6-b502-b3fc3694867e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:32.123360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f275d-7d60-42d6-b502-b3fc3694867e service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:32.123640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f275d-7d60-42d6-b502-b3fc3694867e service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:32.123915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f275d-7d60-42d6-b502-b3fc3694867e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:32.125562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b43f275d-7d60-42d6-b502-b3fc3694867e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:32.216375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f275d-7d60-42d6-b502-b3fc3694867e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:32.218013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f275d-7d60-42d6-b502-b3fc3694867e service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:32.236518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 717/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:33.096349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba42fe7-680a-4f38-a8f4-3755aaf15f20 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:33.096934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba42fe7-680a-4f38-a8f4-3755aaf15f20 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:33.096934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba42fe7-680a-4f38-a8f4-3755aaf15f20 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:33.098355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:33.103677 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fbdce-a75c-4ea3-b6df-66daab4c5b64 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:33.104121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fbdce-a75c-4ea3-b6df-66daab4c5b64 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:33.104236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fbdce-a75c-4ea3-b6df-66daab4c5b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:33.124257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:33.179169 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-8e0fbdce-a75c-4ea3-b6df-66daab4c5b64 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:33.180372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8e0fbdce-a75c-4ea3-b6df-66daab4c5b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:33.274906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:33.428746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba42fe7-680a-4f38-a8f4-3755aaf15f20 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:33.430221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-aba42fe7-680a-4f38-a8f4-3755aaf15f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:33.537969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:33.572665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba42fe7-680a-4f38-a8f4-3755aaf15f20 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:33.683768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069140a-4d75-437c-ab24-5895ed217bb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:33.684322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069140a-4d75-437c-ab24-5895ed217bb9 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:33.684602 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069140a-4d75-437c-ab24-5895ed217bb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:33.684903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069140a-4d75-437c-ab24-5895ed217bb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:33.686547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3069140a-4d75-437c-ab24-5895ed217bb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:33.763644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:33.777475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069140a-4d75-437c-ab24-5895ed217bb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:33.779171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069140a-4d75-437c-ab24-5895ed217bb9 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:33.796988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 707/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:34.658350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fe4d6-1129-4ba5-aba4-cbaf238aa89b None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:34.658932 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fe4d6-1129-4ba5-aba4-cbaf238aa89b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:34.658932 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fe4d6-1129-4ba5-aba4-cbaf238aa89b None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:34.660214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:34.665738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7bb11-de3f-441e-ba4b-0889506a61a5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:34.666171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7bb11-de3f-441e-ba4b-0889506a61a5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:34.666284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7bb11-de3f-441e-ba4b-0889506a61a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:34.742089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4cc7bb11-de3f-441e-ba4b-0889506a61a5 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:34.743303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4cc7bb11-de3f-441e-ba4b-0889506a61a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:34.840049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:35.001978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65fe4d6-1129-4ba5-aba4-cbaf238aa89b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:35.003447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b65fe4d6-1129-4ba5-aba4-cbaf238aa89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:35.086515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:35.147842 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65fe4d6-1129-4ba5-aba4-cbaf238aa89b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:35.257539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bbd974-8602-4f92-8c27-02b179519336 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:35.258067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbd974-8602-4f92-8c27-02b179519336 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:35.258342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbd974-8602-4f92-8c27-02b179519336 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:35.258610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bbd974-8602-4f92-8c27-02b179519336 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:35.260200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-85bbd974-8602-4f92-8c27-02b179519336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:35.308229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:35.349274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbd974-8602-4f92-8c27-02b179519336 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:35.351303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bbd974-8602-4f92-8c27-02b179519336 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:35.368640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 720/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:36.236632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe031b9c-38c2-4567-b6b5-5e43382eb1d3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:36.237184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe031b9c-38c2-4567-b6b5-5e43382eb1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:36.237184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe031b9c-38c2-4567-b6b5-5e43382eb1d3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:36.238545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:36.243738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6ed75-03d3-40cd-b143-602d6263414f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:36.244151 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6ed75-03d3-40cd-b143-602d6263414f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:36.244286 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6ed75-03d3-40cd-b143-602d6263414f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:36.319893 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-63a6ed75-03d3-40cd-b143-602d6263414f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:36.321130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-63a6ed75-03d3-40cd-b143-602d6263414f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:36.416321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:36.569515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe031b9c-38c2-4567-b6b5-5e43382eb1d3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:36.576132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fe031b9c-38c2-4567-b6b5-5e43382eb1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:36.620185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:36.714538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe031b9c-38c2-4567-b6b5-5e43382eb1d3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:36.825743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf20539-b897-4cad-a824-0c153efe07cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:36.826337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf20539-b897-4cad-a824-0c153efe07cc service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:36.826618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf20539-b897-4cad-a824-0c153efe07cc service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:36.826891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf20539-b897-4cad-a824-0c153efe07cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:36.828523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8bf20539-b897-4cad-a824-0c153efe07cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:36.836018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:36.922380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf20539-b897-4cad-a824-0c153efe07cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:36.923964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf20539-b897-4cad-a824-0c153efe07cc service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:36.942429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 710/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:37.853758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c58e6-afe0-4ed9-9bfc-ac63dbc4e6f1 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:37.854340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c58e6-afe0-4ed9-9bfc-ac63dbc4e6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:37.854340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c58e6-afe0-4ed9-9bfc-ac63dbc4e6f1 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:37.855642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:37.860732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02387ed-d8d3-4f30-bc72-18427fa62f18 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:37.861181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02387ed-d8d3-4f30-bc72-18427fa62f18 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:37.861313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02387ed-d8d3-4f30-bc72-18427fa62f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:37.936112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e02387ed-d8d3-4f30-bc72-18427fa62f18 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:37.937355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e02387ed-d8d3-4f30-bc72-18427fa62f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:38.004136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:38.032780 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:38.182948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c58e6-afe0-4ed9-9bfc-ac63dbc4e6f1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:38.184376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5d6c58e6-afe0-4ed9-9bfc-ac63dbc4e6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:38.201639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:38.327044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c58e6-afe0-4ed9-9bfc-ac63dbc4e6f1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:38.410435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:38.435914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d6331-2933-46eb-a310-403af230aa4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:38.436436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d6331-2933-46eb-a310-403af230aa4e service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:38.436712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d6331-2933-46eb-a310-403af230aa4e service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:38.436977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d6331-2933-46eb-a310-403af230aa4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:38.438595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-158d6331-2933-46eb-a310-403af230aa4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:38.525298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d6331-2933-46eb-a310-403af230aa4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:38.526887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d6331-2933-46eb-a310-403af230aa4e service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:38.544982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 723/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:39.415679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacf28e-7a3b-48b9-a6bc-03570b36d23f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:39.416205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacf28e-7a3b-48b9-a6bc-03570b36d23f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:39.416205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacf28e-7a3b-48b9-a6bc-03570b36d23f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:39.417641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:39.422607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943c034-199d-48d4-9785-e1ee451697a3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:39.423293 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943c034-199d-48d4-9785-e1ee451697a3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:39.423293 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943c034-199d-48d4-9785-e1ee451697a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:39.499967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7943c034-199d-48d4-9785-e1ee451697a3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:39.501219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7943c034-199d-48d4-9785-e1ee451697a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:39.542524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:39.600517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:53:39.749900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacf28e-7a3b-48b9-a6bc-03570b36d23f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:39.751519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1cacf28e-7a3b-48b9-a6bc-03570b36d23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:39.895424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacf28e-7a3b-48b9-a6bc-03570b36d23f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:39.949490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:40.007747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e85c06-4bca-4564-84e3-22c8a40007d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:40.008281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e85c06-4bca-4564-84e3-22c8a40007d7 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:40.008562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e85c06-4bca-4564-84e3-22c8a40007d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:40.008841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e85c06-4bca-4564-84e3-22c8a40007d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:40.010473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-43e85c06-4bca-4564-84e3-22c8a40007d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:40.099060 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e85c06-4bca-4564-84e3-22c8a40007d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:40.101292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e85c06-4bca-4564-84e3-22c8a40007d7 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:40.118906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 713/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:40.985784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0f417-6dbb-4ac7-9d57-df9a2e28626c None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:40.986345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0f417-6dbb-4ac7-9d57-df9a2e28626c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:40.986345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0f417-6dbb-4ac7-9d57-df9a2e28626c None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:40.987691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:40.992909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e110f6b-0aae-4bed-a7ba-ec16efab1199 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:40.993363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e110f6b-0aae-4bed-a7ba-ec16efab1199 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:40.993489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e110f6b-0aae-4bed-a7ba-ec16efab1199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:41.068670 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0e110f6b-0aae-4bed-a7ba-ec16efab1199 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:41.069862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0e110f6b-0aae-4bed-a7ba-ec16efab1199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:41.071895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:41.166867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:41.314427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0f417-6dbb-4ac7-9d57-df9a2e28626c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:41.315957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3aa0f417-6dbb-4ac7-9d57-df9a2e28626c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:41.461233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0f417-6dbb-4ac7-9d57-df9a2e28626c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:41.486981 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:41.571955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a60cd-5153-4c54-bd3f-33e9c9d1bf30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:41.572552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a60cd-5153-4c54-bd3f-33e9c9d1bf30 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:41.572827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a60cd-5153-4c54-bd3f-33e9c9d1bf30 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:41.573125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a60cd-5153-4c54-bd3f-33e9c9d1bf30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:41.574785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e77a60cd-5153-4c54-bd3f-33e9c9d1bf30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:41.665323 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a60cd-5153-4c54-bd3f-33e9c9d1bf30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:41.666911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a60cd-5153-4c54-bd3f-33e9c9d1bf30 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:41.685132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 726/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:43.594527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a43e40-875b-4457-a0c8-816cf403604b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:43.595081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a43e40-875b-4457-a0c8-816cf403604b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:43.595081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a43e40-875b-4457-a0c8-816cf403604b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:43.596429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:43.602158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31217cdf-78f0-4c2e-a82a-d3fb23febb1d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:43.602538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31217cdf-78f0-4c2e-a82a-d3fb23febb1d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:43.602707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31217cdf-78f0-4c2e-a82a-d3fb23febb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:43.643711 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:43.668211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-31217cdf-78f0-4c2e-a82a-d3fb23febb1d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:43.671747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-31217cdf-78f0-4c2e-a82a-d3fb23febb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:43.768122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:43.807561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a43e40-875b-4457-a0c8-816cf403604b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:43.809147 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d0a43e40-875b-4457-a0c8-816cf403604b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:43.932246 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a43e40-875b-4457-a0c8-816cf403604b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:43.964635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:44.006278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c8047-50b3-49d3-b8fd-a1380daf3e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:44.006889 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c8047-50b3-49d3-b8fd-a1380daf3e50 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:44.007170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c8047-50b3-49d3-b8fd-a1380daf3e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:44.007445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c8047-50b3-49d3-b8fd-a1380daf3e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:44.009075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-054c8047-50b3-49d3-b8fd-a1380daf3e50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:44.117685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c8047-50b3-49d3-b8fd-a1380daf3e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:44.119105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c8047-50b3-49d3-b8fd-a1380daf3e50 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:44.137878 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 716/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:46.278926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f04fb4-5060-431e-8cf2-64e711bc3138 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:46.279454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f04fb4-5060-431e-8cf2-64e711bc3138 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:46.279454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f04fb4-5060-431e-8cf2-64e711bc3138 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:46.280829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:46.286184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368ebbb-6924-49f1-b91c-b9d4eb70c6b1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:46.286736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368ebbb-6924-49f1-b91c-b9d4eb70c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:46.287024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368ebbb-6924-49f1-b91c-b9d4eb70c6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:46.352570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d368ebbb-6924-49f1-b91c-b9d4eb70c6b1 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:46.353800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d368ebbb-6924-49f1-b91c-b9d4eb70c6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:46.366841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:46.449938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:46.488722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f04fb4-5060-431e-8cf2-64e711bc3138 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:46.491074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-82f04fb4-5060-431e-8cf2-64e711bc3138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:46.594505 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f04fb4-5060-431e-8cf2-64e711bc3138 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:46.607028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:46.670044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35afc06e-b807-4041-9b70-e355d4b2a2b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:46.670644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afc06e-b807-4041-9b70-e355d4b2a2b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:46.670916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afc06e-b807-4041-9b70-e355d4b2a2b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:46.671183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35afc06e-b807-4041-9b70-e355d4b2a2b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:46.672782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-35afc06e-b807-4041-9b70-e355d4b2a2b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:46.782014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afc06e-b807-4041-9b70-e355d4b2a2b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:46.783451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35afc06e-b807-4041-9b70-e355d4b2a2b5 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:46.801849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 729/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:49.156900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216241-1fa6-4746-97c6-77a1b15e4a0c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:49.157531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216241-1fa6-4746-97c6-77a1b15e4a0c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:49.157531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216241-1fa6-4746-97c6-77a1b15e4a0c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:49.158851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:49.164615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32620-a7d5-4064-b6e9-77e7697b93e1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:49.165110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32620-a7d5-4064-b6e9-77e7697b93e1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:49.165218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32620-a7d5-4064-b6e9-77e7697b93e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:49.212952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:49.231380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a7d32620-a7d5-4064-b6e9-77e7697b93e1 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:49.232751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a7d32620-a7d5-4064-b6e9-77e7697b93e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:49.328984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:49.368689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216241-1fa6-4746-97c6-77a1b15e4a0c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:49.370284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-69216241-1fa6-4746-97c6-77a1b15e4a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:49.473444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216241-1fa6-4746-97c6-77a1b15e4a0c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:49.512271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:49.545437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887ea47-1854-439a-aa44-679603b3c0b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:49.546038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887ea47-1854-439a-aa44-679603b3c0b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:49.546319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887ea47-1854-439a-aa44-679603b3c0b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:49.546594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887ea47-1854-439a-aa44-679603b3c0b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:49.548188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1887ea47-1854-439a-aa44-679603b3c0b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:49.656400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887ea47-1854-439a-aa44-679603b3c0b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:49.657831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887ea47-1854-439a-aa44-679603b3c0b0 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:49.676838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 719/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:51.796763 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e584f-5461-4219-abff-c2b395217c41 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:51.797354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e584f-5461-4219-abff-c2b395217c41 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:51.797354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e584f-5461-4219-abff-c2b395217c41 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:51.798724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:51.804692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3671650-1ed7-4ab2-9cfe-ae27d10166a9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:51.805130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3671650-1ed7-4ab2-9cfe-ae27d10166a9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:51.805293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3671650-1ed7-4ab2-9cfe-ae27d10166a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:51.870797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b3671650-1ed7-4ab2-9cfe-ae27d10166a9 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:51.871994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b3671650-1ed7-4ab2-9cfe-ae27d10166a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:51.892590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:51.968984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:52.013760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e584f-5461-4219-abff-c2b395217c41 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:52.015303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-772e584f-5461-4219-abff-c2b395217c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:52.119148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e584f-5461-4219-abff-c2b395217c41 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:52.138413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:52.191876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b35f0-bd81-4b40-846a-ff9f25c9c77a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:52.192534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b35f0-bd81-4b40-846a-ff9f25c9c77a service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:52.192844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b35f0-bd81-4b40-846a-ff9f25c9c77a service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:52.193140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b35f0-bd81-4b40-846a-ff9f25c9c77a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:52.194741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-311b35f0-bd81-4b40-846a-ff9f25c9c77a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:52.303002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b35f0-bd81-4b40-846a-ff9f25c9c77a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:52.304483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b35f0-bd81-4b40-846a-ff9f25c9c77a service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:52.321945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 732/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:54.373435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fd72d-5189-4396-ac78-1107d19b0a3f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:54.374001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fd72d-5189-4396-ac78-1107d19b0a3f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:54.374001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fd72d-5189-4396-ac78-1107d19b0a3f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:54.375333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:54.381146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3b4ae-7be0-4832-9732-452a825ee5ab None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:54.381543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3b4ae-7be0-4832-9732-452a825ee5ab None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:54.381692 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3b4ae-7be0-4832-9732-452a825ee5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:54.444020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:54.446565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-38a3b4ae-7be0-4832-9732-452a825ee5ab None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:54.447784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-38a3b4ae-7be0-4832-9732-452a825ee5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:54.542062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:54.580965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fd72d-5189-4396-ac78-1107d19b0a3f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:54.582642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fe7fd72d-5189-4396-ac78-1107d19b0a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:54.686100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fd72d-5189-4396-ac78-1107d19b0a3f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:54.735923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:54.760514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75128f44-dc70-4515-a854-1f6ec013e406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:54.761178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75128f44-dc70-4515-a854-1f6ec013e406 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:54.761459 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75128f44-dc70-4515-a854-1f6ec013e406 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:54.761730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75128f44-dc70-4515-a854-1f6ec013e406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:54.763323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-75128f44-dc70-4515-a854-1f6ec013e406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:54.870841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75128f44-dc70-4515-a854-1f6ec013e406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:54.872251 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75128f44-dc70-4515-a854-1f6ec013e406 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:54.890390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 722/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:56.931254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16af349-80b0-4bbd-a844-57874dbf8bf6 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:56.931791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16af349-80b0-4bbd-a844-57874dbf8bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:56.931791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16af349-80b0-4bbd-a844-57874dbf8bf6 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:56.933088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:56.938233 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e402be-52d0-423e-9131-6cc1a9ade63f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:56.938684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e402be-52d0-423e-9131-6cc1a9ade63f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:56.938790 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e402be-52d0-423e-9131-6cc1a9ade63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:57.004651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-50e402be-52d0-423e-9131-6cc1a9ade63f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:57.006384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-50e402be-52d0-423e-9131-6cc1a9ade63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:57.035118 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:57.102754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:57.141134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16af349-80b0-4bbd-a844-57874dbf8bf6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:57.142659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a16af349-80b0-4bbd-a844-57874dbf8bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:57.248930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16af349-80b0-4bbd-a844-57874dbf8bf6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:57.278456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:57.324193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c826b85-18ec-4379-afe6-38cd54803b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:57.324799 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c826b85-18ec-4379-afe6-38cd54803b32 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:57.325107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c826b85-18ec-4379-afe6-38cd54803b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:57.325385 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c826b85-18ec-4379-afe6-38cd54803b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:57.326993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8c826b85-18ec-4379-afe6-38cd54803b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:57.435249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c826b85-18ec-4379-afe6-38cd54803b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:53:57.436742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c826b85-18ec-4379-afe6-38cd54803b32 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:53:57.454528 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 735/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:53:59.516764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51158cd-5317-4ea2-b279-7fd8a798946b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:59.517310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51158cd-5317-4ea2-b279-7fd8a798946b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:59.517310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51158cd-5317-4ea2-b279-7fd8a798946b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:59.518603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:53:59.524427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3f3a3-7ee4-4d46-a8b6-c0fe4c98dd97 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:59.524846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3f3a3-7ee4-4d46-a8b6-c0fe4c98dd97 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:59.525038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3f3a3-7ee4-4d46-a8b6-c0fe4c98dd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:59.595320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0fb3f3a3-7ee4-4d46-a8b6-c0fe4c98dd97 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:53:59.596555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0fb3f3a3-7ee4-4d46-a8b6-c0fe4c98dd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:59.605707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:59.705553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:53:59.743489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51158cd-5317-4ea2-b279-7fd8a798946b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:53:59.745077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b51158cd-5317-4ea2-b279-7fd8a798946b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:53:59.847676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51158cd-5317-4ea2-b279-7fd8a798946b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:53:59.903266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:53:59.921534 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85933aac-85fc-4c29-86c7-f87d6af3ca0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:53:59.922145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85933aac-85fc-4c29-86c7-f87d6af3ca0b service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:53:59.922425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85933aac-85fc-4c29-86c7-f87d6af3ca0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:53:59.922717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85933aac-85fc-4c29-86c7-f87d6af3ca0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:53:59.924301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-85933aac-85fc-4c29-86c7-f87d6af3ca0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:00.032111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85933aac-85fc-4c29-86c7-f87d6af3ca0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:54:00.033556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85933aac-85fc-4c29-86c7-f87d6af3ca0b service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:54:00.051587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 725/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:53:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:54:02.130495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ec2f-c0bc-45cc-a632-1088a9989343 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:02.131163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ec2f-c0bc-45cc-a632-1088a9989343 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:02.131163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ec2f-c0bc-45cc-a632-1088a9989343 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:02.132386 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:54:02.137728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7f952-662c-4dc1-8956-5d965092a6f8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:02.138180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7f952-662c-4dc1-8956-5d965092a6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:02.138295 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7f952-662c-4dc1-8956-5d965092a6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:02.205154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ccb7f952-662c-4dc1-8956-5d965092a6f8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:54:02.206366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ccb7f952-662c-4dc1-8956-5d965092a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:02.245039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:02.303521 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:54:02.341037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034ec2f-c0bc-45cc-a632-1088a9989343 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:54:02.342582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d034ec2f-c0bc-45cc-a632-1088a9989343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:02.448198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034ec2f-c0bc-45cc-a632-1088a9989343 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:54:02.485878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:02.522647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30466b3c-1427-42dd-b961-875ee4ed8c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:54:02.523249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30466b3c-1427-42dd-b961-875ee4ed8c3b service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:02.523531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30466b3c-1427-42dd-b961-875ee4ed8c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:02.523807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30466b3c-1427-42dd-b961-875ee4ed8c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:02.525448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-30466b3c-1427-42dd-b961-875ee4ed8c3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:02.633103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30466b3c-1427-42dd-b961-875ee4ed8c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:54:02.634539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30466b3c-1427-42dd-b961-875ee4ed8c3b service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:54:02.653294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 738/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:54:04.675016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b68b3-c74f-4218-84e4-15d5dd39bdfe None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:04.675633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b68b3-c74f-4218-84e4-15d5dd39bdfe None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:04.675633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b68b3-c74f-4218-84e4-15d5dd39bdfe None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:04.677562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:54:04.683173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb7e67-017e-4636-a786-d2ea327607bd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:04.683630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb7e67-017e-4636-a786-d2ea327607bd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:04.683738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb7e67-017e-4636-a786-d2ea327607bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:04.750031 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e0eb7e67-017e-4636-a786-d2ea327607bd None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:54:04.751265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e0eb7e67-017e-4636-a786-d2ea327607bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:04.768664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:04.971503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:54:05.009567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b68b3-c74f-4218-84e4-15d5dd39bdfe None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:54:05.011099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cc1b68b3-c74f-4218-84e4-15d5dd39bdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:05.115568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b68b3-c74f-4218-84e4-15d5dd39bdfe None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:54:05.182740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:05.189481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c69c2b-6af3-45a8-8433-a88871b3b212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:54:05.191574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c69c2b-6af3-45a8-8433-a88871b3b212 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:05.191574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c69c2b-6af3-45a8-8433-a88871b3b212 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:05.191574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c69c2b-6af3-45a8-8433-a88871b3b212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:05.192182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a3c69c2b-6af3-45a8-8433-a88871b3b212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:05.301151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c69c2b-6af3-45a8-8433-a88871b3b212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:54:05.303401 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c69c2b-6af3-45a8-8433-a88871b3b212 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:54:05.321369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 728/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:54:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:54:07.334990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d844e9a-d55b-431f-ab4c-9e73f584e9ed None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:07.335514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d844e9a-d55b-431f-ab4c-9e73f584e9ed None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:07.335514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d844e9a-d55b-431f-ab4c-9e73f584e9ed None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:07.336818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:54:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:54:07.342883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bc696-cde2-4a57-80ec-bae09f25da4e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:07.343334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bc696-cde2-4a57-80ec-bae09f25da4e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:07.343449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bc696-cde2-4a57-80ec-bae09f25da4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:07.410084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-958bc696-cde2-4a57-80ec-bae09f25da4e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:54:07.411407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-958bc696-cde2-4a57-80ec-bae09f25da4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:07.460333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:07.506431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:54:07.542928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d844e9a-d55b-431f-ab4c-9e73f584e9ed None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:54:07.544443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1d844e9a-d55b-431f-ab4c-9e73f584e9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:07.645670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d844e9a-d55b-431f-ab4c-9e73f584e9ed None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:54:07.691212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:07.717152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305295c-4796-436d-a2b4-8211026f77e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:54:07.717687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305295c-4796-436d-a2b4-8211026f77e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:07.717960 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305295c-4796-436d-a2b4-8211026f77e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:07.718227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305295c-4796-436d-a2b4-8211026f77e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:07.719806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a305295c-4796-436d-a2b4-8211026f77e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:07.824821 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305295c-4796-436d-a2b4-8211026f77e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:54:07.826261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305295c-4796-436d-a2b4-8211026f77e2 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:54:07.844096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 741/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:54:17.504248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=22 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:17.841640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=66 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:27.328593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4eb2a-d73e-435c-a38e-c78f17612b43 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:27.329124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4eb2a-d73e-435c-a38e-c78f17612b43 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:27.329232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4eb2a-d73e-435c-a38e-c78f17612b43 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:27.330676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 730/1471] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:54:27.335942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eda78-3e00-462b-8d14-504073d0fa5e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:27.336367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eda78-3e00-462b-8d14-504073d0fa5e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:27.336485 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eda78-3e00-462b-8d14-504073d0fa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:27.400770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-eb2eda78-3e00-462b-8d14-504073d0fa5e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:54:27.402015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-eb2eda78-3e00-462b-8d14-504073d0fa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:27.498890 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 742/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:54:27.583289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43589a7e-b116-436b-98f0-d6d15923fc8d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:27.583570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43589a7e-b116-436b-98f0-d6d15923fc8d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:27.583845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43589a7e-b116-436b-98f0-d6d15923fc8d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:27.585514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 731/1473] 10.222.0.17 () {54 vars in 967 bytes} [Thu Aug 11 09:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:54:27.590227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263f545-6225-496e-ba52-cee198451fbc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:27.590514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263f545-6225-496e-ba52-cee198451fbc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:27.590791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263f545-6225-496e-ba52-cee198451fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:27.665397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6263f545-6225-496e-ba52-cee198451fbc None None] MFA Rules not processed for user `d83448e4c6704f7c8771132bd67eaa1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:54:27.666615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6263f545-6225-496e-ba52-cee198451fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:27.686822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:27.740693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 743/1474] 10.222.0.17 () {58 vars in 1064 bytes} [Thu Aug 11 09:54:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:54:27.748117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43589a7e-b116-436b-98f0-d6d15923fc8d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:54:27.749723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-43589a7e-b116-436b-98f0-d6d15923fc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:27.851229 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43589a7e-b116-436b-98f0-d6d15923fc8d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:54:27.927429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57aa2a-3e48-459a-8c78-dc620aa3b620 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83448e4c6704f7c8771132bd67eaa1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:54:27.928082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57aa2a-3e48-459a-8c78-dc620aa3b620 service glance] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:27.928354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57aa2a-3e48-459a-8c78-dc620aa3b620 service glance] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:27.928627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57aa2a-3e48-459a-8c78-dc620aa3b620 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:27.930384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0a57aa2a-3e48-459a-8c78-dc620aa3b620 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:27.983916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:28.038871 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57aa2a-3e48-459a-8c78-dc620aa3b620 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:54:28.040284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57aa2a-3e48-459a-8c78-dc620aa3b620 service glance] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:54:28.058269 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 732/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:54:30.215273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e2d3b-7444-4a0e-83ad-4ff7aecc4fb7 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:30.215776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e2d3b-7444-4a0e-83ad-4ff7aecc4fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:30.215883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e2d3b-7444-4a0e-83ad-4ff7aecc4fb7 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:30.217176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 744/1476] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:54:30.222185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99e76e-3947-4a85-a064-3cc427b1fb6b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:30.223207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99e76e-3947-4a85-a064-3cc427b1fb6b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:30.223320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99e76e-3947-4a85-a064-3cc427b1fb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:30.289998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9e99e76e-3947-4a85-a064-3cc427b1fb6b None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:54:30.291211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9e99e76e-3947-4a85-a064-3cc427b1fb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:30.386627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 733/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:54:30.409457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e2d3b-7444-4a0e-83ad-4ff7aecc4fb7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:54:30.411618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-609e2d3b-7444-4a0e-83ad-4ff7aecc4fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:30.533405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e2d3b-7444-4a0e-83ad-4ff7aecc4fb7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:54:30.581178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:30.625813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da057a7a-1bfe-48dd-bb50-4ae6fbb59b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:54:30.626414 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da057a7a-1bfe-48dd-bb50-4ae6fbb59b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:30.626691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da057a7a-1bfe-48dd-bb50-4ae6fbb59b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:30.627000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da057a7a-1bfe-48dd-bb50-4ae6fbb59b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:30.627659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da057a7a-1bfe-48dd-bb50-4ae6fbb59b92 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:54:30.630226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-da057a7a-1bfe-48dd-bb50-4ae6fbb59b92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:54:30.630226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 11 09:54:30.630226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: Aug 11 09:54:30.631350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da057a7a-1bfe-48dd-bb50-4ae6fbb59b92 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:54:30.650654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 745/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Aug 11 09:54:30 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:54:32.676585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1b1d-d345-47f9-b2e3-b3e50265d34b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:32.677260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1b1d-d345-47f9-b2e3-b3e50265d34b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:32.677260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1b1d-d345-47f9-b2e3-b3e50265d34b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:32.679249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 734/1479] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:54:32.684643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafdc5-b050-4db9-9499-4731f5e4ac5a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:32.685101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafdc5-b050-4db9-9499-4731f5e4ac5a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:32.685249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bafdc5-b050-4db9-9499-4731f5e4ac5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:32.753565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e5bafdc5-b050-4db9-9499-4731f5e4ac5a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:54:32.754782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e5bafdc5-b050-4db9-9499-4731f5e4ac5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:32.849721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 746/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:54:32.892472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d1b1d-d345-47f9-b2e3-b3e50265d34b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:54:32.894688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-908d1b1d-d345-47f9-b2e3-b3e50265d34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:32.904504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:33.044639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d1b1d-d345-47f9-b2e3-b3e50265d34b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:54:33.123289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:33.156717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784ad9-27d7-46ad-91bb-468bf7783eef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:54:33.157345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784ad9-27d7-46ad-91bb-468bf7783eef service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:33.157625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784ad9-27d7-46ad-91bb-468bf7783eef service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:33.157899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784ad9-27d7-46ad-91bb-468bf7783eef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:33.159501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5d784ad9-27d7-46ad-91bb-468bf7783eef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:33.248786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d784ad9-27d7-46ad-91bb-468bf7783eef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:54:33.250389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d784ad9-27d7-46ad-91bb-468bf7783eef service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:54:33.268717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 735/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:54:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:54:35.428529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1ab4f-caaa-4e1c-bb01-63b169b8d0d0 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:35.429122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1ab4f-caaa-4e1c-bb01-63b169b8d0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:35.429233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1ab4f-caaa-4e1c-bb01-63b169b8d0d0 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:35.430504 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 747/1482] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:54:35.435577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34701b6f-06d3-4160-8bfd-315dc16ec039 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:35.435997 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34701b6f-06d3-4160-8bfd-315dc16ec039 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:35.436158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34701b6f-06d3-4160-8bfd-315dc16ec039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:35.502641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-34701b6f-06d3-4160-8bfd-315dc16ec039 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:54:35.504593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-34701b6f-06d3-4160-8bfd-315dc16ec039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:35.549726 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:35.599691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 736/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:54:35.636202 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1ab4f-caaa-4e1c-bb01-63b169b8d0d0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:54:35.637763 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-94b1ab4f-caaa-4e1c-bb01-63b169b8d0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:35.661673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:35.739784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1ab4f-caaa-4e1c-bb01-63b169b8d0d0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:54:35.814731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996a0ae-5bd8-43d8-bbab-95df46a3fb36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:54:35.815326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996a0ae-5bd8-43d8-bbab-95df46a3fb36 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:35.815601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996a0ae-5bd8-43d8-bbab-95df46a3fb36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:35.815898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996a0ae-5bd8-43d8-bbab-95df46a3fb36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:35.817597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4996a0ae-5bd8-43d8-bbab-95df46a3fb36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:35.890118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:35.924099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996a0ae-5bd8-43d8-bbab-95df46a3fb36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:54:35.925945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996a0ae-5bd8-43d8-bbab-95df46a3fb36 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:54:35.944834 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 748/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:54:37.990654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e03276-50b4-49bd-b1a9-4aadabf62af2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:37.991202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e03276-50b4-49bd-b1a9-4aadabf62af2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:37.991202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e03276-50b4-49bd-b1a9-4aadabf62af2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:37.992547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 737/1485] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:54:37.997913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53631f4b-5605-4b54-ba69-aa94b0217c34 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:37.998308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53631f4b-5605-4b54-ba69-aa94b0217c34 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:37.998450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53631f4b-5605-4b54-ba69-aa94b0217c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:38.064336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-53631f4b-5605-4b54-ba69-aa94b0217c34 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:54:38.065623 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-53631f4b-5605-4b54-ba69-aa94b0217c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:38.160934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 749/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:54:38.199074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e03276-50b4-49bd-b1a9-4aadabf62af2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:54:38.200559 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b3e03276-50b4-49bd-b1a9-4aadabf62af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:38.304625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e03276-50b4-49bd-b1a9-4aadabf62af2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:54:38.378065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:38.380487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc7357-4c9b-4b51-b505-cff813c1f0fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:54:38.381135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7357-4c9b-4b51-b505-cff813c1f0fd service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:38.381418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7357-4c9b-4b51-b505-cff813c1f0fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:38.381692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc7357-4c9b-4b51-b505-cff813c1f0fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:38.383312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e6dc7357-4c9b-4b51-b505-cff813c1f0fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:38.492837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc7357-4c9b-4b51-b505-cff813c1f0fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:54:38.494818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc7357-4c9b-4b51-b505-cff813c1f0fd service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:54:38.513050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 738/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:54:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:54:40.585583 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368a57-9379-44d1-8f85-5fe36d6f1519 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:40.586119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368a57-9379-44d1-8f85-5fe36d6f1519 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:40.586119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368a57-9379-44d1-8f85-5fe36d6f1519 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:40.587434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 750/1488] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:54:40.592545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee97c4-bad2-4c21-b132-c96133052c32 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:40.592948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee97c4-bad2-4c21-b132-c96133052c32 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:40.593127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee97c4-bad2-4c21-b132-c96133052c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:40.658947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-95ee97c4-bad2-4c21-b132-c96133052c32 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:54:40.660259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-95ee97c4-bad2-4c21-b132-c96133052c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:40.712236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:40.757527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 739/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:54:40.796427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368a57-9379-44d1-8f85-5fe36d6f1519 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:54:40.798004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-be368a57-9379-44d1-8f85-5fe36d6f1519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:40.834086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:40.900472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368a57-9379-44d1-8f85-5fe36d6f1519 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:54:40.972554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841443a-9edc-4c5f-9381-c1ca72b23a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:54:40.973201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841443a-9edc-4c5f-9381-c1ca72b23a9d service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:40.973479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841443a-9edc-4c5f-9381-c1ca72b23a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:40.973750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841443a-9edc-4c5f-9381-c1ca72b23a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:40.975338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3841443a-9edc-4c5f-9381-c1ca72b23a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:41.054333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:41.083708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841443a-9edc-4c5f-9381-c1ca72b23a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:54:41.084674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841443a-9edc-4c5f-9381-c1ca72b23a9d service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:54:41.103418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 751/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:54:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:54:43.448206 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e9d7e-ee08-4f46-9dd8-47c868bc165b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:43.448735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e9d7e-ee08-4f46-9dd8-47c868bc165b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:43.448735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e9d7e-ee08-4f46-9dd8-47c868bc165b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:43.450067 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 740/1491] 10.222.0.17 () {54 vars in 927 bytes} [Thu Aug 11 09:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:54:43.455094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c18d5-610a-4bd4-b340-659ffc5b52dc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:43.455488 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c18d5-610a-4bd4-b340-659ffc5b52dc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:43.455681 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c18d5-610a-4bd4-b340-659ffc5b52dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:43.520959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-657c18d5-610a-4bd4-b340-659ffc5b52dc None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:54:43.522199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-657c18d5-610a-4bd4-b340-659ffc5b52dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:43.617578 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 752/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Thu Aug 11 09:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:54:43.657532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e9d7e-ee08-4f46-9dd8-47c868bc165b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:54:43.659079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5d0e9d7e-ee08-4f46-9dd8-47c868bc165b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:43.764324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e9d7e-ee08-4f46-9dd8-47c868bc165b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:54:43.837797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542543b4-7cfc-4dca-8936-7ae9b58af9bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:54:43.860530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542543b4-7cfc-4dca-8936-7ae9b58af9bf service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:54:43.860530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542543b4-7cfc-4dca-8936-7ae9b58af9bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:54:43.860530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542543b4-7cfc-4dca-8936-7ae9b58af9bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:54:43.860530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-542543b4-7cfc-4dca-8936-7ae9b58af9bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:54:43.860530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:43.948886 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542543b4-7cfc-4dca-8936-7ae9b58af9bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:54:43.950325 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542543b4-7cfc-4dca-8936-7ae9b58af9bf service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:54:43.968806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 741/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:54:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:54:53.615809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=82 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:54:53.966495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=50 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:04.288709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0cbba-edf1-4a46-a0e1-e5f11afef4bd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:04.289306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0cbba-edf1-4a46-a0e1-e5f11afef4bd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:04.289306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0cbba-edf1-4a46-a0e1-e5f11afef4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:04.367038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5ee0cbba-edf1-4a46-a0e1-e5f11afef4bd None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:56:04.374260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5ee0cbba-edf1-4a46-a0e1-e5f11afef4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:04.466334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 753/1494] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:56:04.474614 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165487a-58e6-4947-b663-f6bb3f638e6c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:04.475125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165487a-58e6-4947-b663-f6bb3f638e6c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:04.475229 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165487a-58e6-4947-b663-f6bb3f638e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:04.549644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5165487a-58e6-4947-b663-f6bb3f638e6c None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:56:04.550845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5165487a-58e6-4947-b663-f6bb3f638e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:04.646048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 742/1495] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:56:04.651076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0cbba-edf1-4a46-a0e1-e5f11afef4bd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:56:04.652273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5ee0cbba-edf1-4a46-a0e1-e5f11afef4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:04.663503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0cbba-edf1-4a46-a0e1-e5f11afef4bd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:56:04.724341 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:04.759176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d1e63-c6fc-482c-ba4d-3525f7c591ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:56:04.759782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d1e63-c6fc-482c-ba4d-3525f7c591ec admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:04.760060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d1e63-c6fc-482c-ba4d-3525f7c591ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:04.760328 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d1e63-c6fc-482c-ba4d-3525f7c591ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:04.760971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d1e63-c6fc-482c-ba4d-3525f7c591ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:56:04.761362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d1e63-c6fc-482c-ba4d-3525f7c591ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:56:04.762376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-1e5d1e63-c6fc-482c-ba4d-3525f7c591ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:56:04.762376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 11 09:56:04.762376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: Aug 11 09:56:04.763248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d1e63-c6fc-482c-ba4d-3525f7c591ec admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:56:04.774389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 754/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:56:04.782767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165487a-58e6-4947-b663-f6bb3f638e6c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:56:04.792242 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165487a-58e6-4947-b663-f6bb3f638e6c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:56:04.855300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:04.890092 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d631094-bbd3-4008-ae85-4107662b7dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:56:04.890696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d631094-bbd3-4008-ae85-4107662b7dde admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:04.890975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d631094-bbd3-4008-ae85-4107662b7dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:04.891248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d631094-bbd3-4008-ae85-4107662b7dde admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:04.891866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d631094-bbd3-4008-ae85-4107662b7dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:56:04.893860 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-8d631094-bbd3-4008-ae85-4107662b7dde admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:56:04.893860 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 11 09:56:04.893860 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: Aug 11 09:56:04.895054 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d631094-bbd3-4008-ae85-4107662b7dde admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:56:05.045443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 743/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:56:04 2022] POST /identity/v3/projects => generated 377 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:56:05.050302 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d1e63-c6fc-482c-ba4d-3525f7c591ec admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:56:05.051863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1e5d1e63-c6fc-482c-ba4d-3525f7c591ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:05.084813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d1e63-c6fc-482c-ba4d-3525f7c591ec admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:56:05.178706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb22a1-c774-4687-80e7-1508f245790e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:56:05.179308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb22a1-c774-4687-80e7-1508f245790e admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:05.179586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb22a1-c774-4687-80e7-1508f245790e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:05.179861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb22a1-c774-4687-80e7-1508f245790e admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:05.180492 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb22a1-c774-4687-80e7-1508f245790e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:56:05.181564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-95fb22a1-c774-4687-80e7-1508f245790e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:56:05.181564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 11 09:56:05.181564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: Aug 11 09:56:05.182842 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb22a1-c774-4687-80e7-1508f245790e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:56:05.394962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 755/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:56:05 2022] POST /identity/v3/users => generated 289 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:56:05.399062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d631094-bbd3-4008-ae85-4107662b7dde admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:56:05.400632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8d631094-bbd3-4008-ae85-4107662b7dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:05.463997 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d631094-bbd3-4008-ae85-4107662b7dde admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:56:05.545491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:05.556010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14320d3-acf2-4ad7-81b6-4112b02af4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:56:05.556616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14320d3-acf2-4ad7-81b6-4112b02af4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:05.556890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14320d3-acf2-4ad7-81b6-4112b02af4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:05.557198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14320d3-acf2-4ad7-81b6-4112b02af4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:05.557840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14320d3-acf2-4ad7-81b6-4112b02af4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:56:05.558833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-c14320d3-acf2-4ad7-81b6-4112b02af4d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:56:05.558833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 11 09:56:05.558833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: Aug 11 09:56:05.559738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14320d3-acf2-4ad7-81b6-4112b02af4d6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:56:05.572388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 744/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:56:05.576522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb22a1-c774-4687-80e7-1508f245790e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:56:05.578062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-95fb22a1-c774-4687-80e7-1508f245790e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:05.589731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb22a1-c774-4687-80e7-1508f245790e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:56:05.614625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:05.683643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5afa6-dbfc-47f5-95b4-aea31d67357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:56:05.684267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5afa6-dbfc-47f5-95b4-aea31d67357e admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:05.684542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5afa6-dbfc-47f5-95b4-aea31d67357e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:05.684807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5afa6-dbfc-47f5-95b4-aea31d67357e admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:05.685466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5afa6-dbfc-47f5-95b4-aea31d67357e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:56:05.686466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-57f5afa6-dbfc-47f5-95b4-aea31d67357e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:56:05.686466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 11 09:56:05.686466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: Aug 11 09:56:05.687330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5afa6-dbfc-47f5-95b4-aea31d67357e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:56:05.700793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 756/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:56:05.705063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14320d3-acf2-4ad7-81b6-4112b02af4d6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:56:05.706677 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c14320d3-acf2-4ad7-81b6-4112b02af4d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:05.740268 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14320d3-acf2-4ad7-81b6-4112b02af4d6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:56:05.832311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f378d4f-a650-4aab-822c-48fa1b03b3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:56:05.832964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f378d4f-a650-4aab-822c-48fa1b03b3eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:05.833267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f378d4f-a650-4aab-822c-48fa1b03b3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:05.833538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f378d4f-a650-4aab-822c-48fa1b03b3eb admin admin] PATH_INFO: `/v3/projects/093ac1adfef94605ad73dcb640fa5010/users/62b5123418f34e7183ddd19023fcd90e/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:05.855814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f378d4f-a650-4aab-822c-48fa1b03b3eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=093ac1adfef94605ad73dcb640fa5010, user_id=62b5123418f34e7183ddd19023fcd90e, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:56:05.856829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-0f378d4f-a650-4aab-822c-48fa1b03b3eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:56:05.856829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 11 09:56:05.856829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: Aug 11 09:56:05.858778 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f378d4f-a650-4aab-822c-48fa1b03b3eb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:56:06.014532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 745/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:56:05 2022] PUT /identity/v3/projects/093ac1adfef94605ad73dcb640fa5010/users/62b5123418f34e7183ddd19023fcd90e/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:56:06.026718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1e0fc-2ca0-494f-b667-68d974a410e4 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:06.027003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1e0fc-2ca0-494f-b667-68d974a410e4 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:06.027274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1e0fc-2ca0-494f-b667-68d974a410e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:06.083423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-56e1e0fc-2ca0-494f-b667-68d974a410e4 None None] MFA Rules not processed for user `62b5123418f34e7183ddd19023fcd90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:56:06.084648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-56e1e0fc-2ca0-494f-b667-68d974a410e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:06.161408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 757/1502] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:56:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:56:06.174473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee2e4-820c-4952-bfde-74198895e312 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:06.174858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee2e4-820c-4952-bfde-74198895e312 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:06.175134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee2e4-820c-4952-bfde-74198895e312 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:06.176465 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 746/1503] 10.222.0.17 () {54 vars in 964 bytes} [Thu Aug 11 09:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:56:06.181275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ccfd-4259-444d-8c58-0f0e2ac5729d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:06.181561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ccfd-4259-444d-8c58-0f0e2ac5729d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:06.181831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70ccfd-4259-444d-8c58-0f0e2ac5729d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:06.214362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:06.258253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1b70ccfd-4259-444d-8c58-0f0e2ac5729d None None] MFA Rules not processed for user `029bb992f9974eec83c0fe7f9fa58826`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:56:06.259453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1b70ccfd-4259-444d-8c58-0f0e2ac5729d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:06.332937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 758/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Thu Aug 11 09:56:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:56:06.342044 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ee2e4-820c-4952-bfde-74198895e312 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:56:06.343552 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ec3ee2e4-820c-4952-bfde-74198895e312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:06.393673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:06.447035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ee2e4-820c-4952-bfde-74198895e312 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:56:06.521432 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695dbe5-b3cd-4564-9fdb-188b59efddd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:56:06.522033 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695dbe5-b3cd-4564-9fdb-188b59efddd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:06.522307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695dbe5-b3cd-4564-9fdb-188b59efddd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:06.522573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695dbe5-b3cd-4564-9fdb-188b59efddd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:06.524154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5695dbe5-b3cd-4564-9fdb-188b59efddd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:06.624130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695dbe5-b3cd-4564-9fdb-188b59efddd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:56:06.625578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695dbe5-b3cd-4564-9fdb-188b59efddd7 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:56:06.629579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:06.644916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 747/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:56:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:56:06.696536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe0e40-dcc3-4dab-9898-2e9305096f64 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:06.696985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe0e40-dcc3-4dab-9898-2e9305096f64 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:06.697112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe0e40-dcc3-4dab-9898-2e9305096f64 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:06.698338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 759/1506] 10.222.0.17 () {52 vars in 830 bytes} [Thu Aug 11 09:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:56:06.701952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695dbe5-b3cd-4564-9fdb-188b59efddd7 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:56:06.703203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5695dbe5-b3cd-4564-9fdb-188b59efddd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:06.801580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695dbe5-b3cd-4564-9fdb-188b59efddd7 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:56:06.897658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:56:06.898209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:06.898493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:06.898767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] PATH_INFO: `/v3/users/62b5123418f34e7183ddd19023fcd90e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:06.911390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62b5123418f34e7183ddd19023fcd90e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:56:06.912399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:56:06.912399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 11 09:56:06.912399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: Aug 11 09:56:06.913645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:56:06.923538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:07.117176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62b5123418f34e7183ddd19023fcd90e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:56:07.117723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62b5123418f34e7183ddd19023fcd90e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:56:07.118154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62b5123418f34e7183ddd19023fcd90e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:56:07.176451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62b5123418f34e7183ddd19023fcd90e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:56:07.200481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62b5123418f34e7183ddd19023fcd90e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:56:07.219950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 748/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 09:56:06 2022] DELETE /identity/v3/users/62b5123418f34e7183ddd19023fcd90e => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:56:07.224483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09514ac6-973c-4583-afbd-2795161e7f79 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:07.224769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09514ac6-973c-4583-afbd-2795161e7f79 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:07.225108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09514ac6-973c-4583-afbd-2795161e7f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:07.280970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-09514ac6-973c-4583-afbd-2795161e7f79 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:56:07.282208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-09514ac6-973c-4583-afbd-2795161e7f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:07.359317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:07.372695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 760/1508] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:56:07.385651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:56:07.386882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:07.493796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af1bde-f6a2-40b9-a05a-081e9036e369 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:56:07.547836 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:07.566993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487cbc3-89f8-4e52-bece-9f8efc6ca528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:56:07.567635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487cbc3-89f8-4e52-bece-9f8efc6ca528 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:07.567918 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487cbc3-89f8-4e52-bece-9f8efc6ca528 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:07.568195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487cbc3-89f8-4e52-bece-9f8efc6ca528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:07.569841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6487cbc3-89f8-4e52-bece-9f8efc6ca528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:07.678862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487cbc3-89f8-4e52-bece-9f8efc6ca528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:56:07.680283 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487cbc3-89f8-4e52-bece-9f8efc6ca528 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:56:07.698428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 749/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:56:08.216797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09514ac6-973c-4583-afbd-2795161e7f79 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:56:08.218079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-09514ac6-973c-4583-afbd-2795161e7f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:56:08.229933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09514ac6-973c-4583-afbd-2795161e7f79 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:56:08.328597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:56:08.329378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:56:08.329660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:56:08.329924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] PATH_INFO: `/v3/projects/093ac1adfef94605ad73dcb640fa5010` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:56:08.340211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=093ac1adfef94605ad73dcb640fa5010)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:56:08.341261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:56:08.341261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 11 09:56:08.341261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: Aug 11 09:56:08.342444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:56:08.484133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093ac1adfef94605ad73dcb640fa5010', 'request_id': 'req-10589d37-00b7-49d1-b388-7feec49c5d87'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:56:08.490260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093ac1adfef94605ad73dcb640fa5010', 'request_id': 'req-10589d37-00b7-49d1-b388-7feec49c5d87'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:56:08.503170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093ac1adfef94605ad73dcb640fa5010', 'request_id': 'req-10589d37-00b7-49d1-b388-7feec49c5d87'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:56:08.587438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] The token cache is being invalidate because project 093ac1adfef94605ad73dcb640fa5010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 09:56:08.587719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:56:08.588147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-10589d37-00b7-49d1-b388-7feec49c5d87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:56:08.589564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 761/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 09:56:08 2022] DELETE /identity/v3/projects/093ac1adfef94605ad73dcb640fa5010 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:56:17.696182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=62 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:56:18.585191 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:57:36.730068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895abc3e-ff23-4132-ad61-1c2d4bb71b63 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:36.730673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895abc3e-ff23-4132-ad61-1c2d4bb71b63 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:36.730673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895abc3e-ff23-4132-ad61-1c2d4bb71b63 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:36.731977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:57:36.737042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223d23b-dcac-4d76-895a-0698065cd4bd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:36.737499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223d23b-dcac-4d76-895a-0698065cd4bd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:36.737617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223d23b-dcac-4d76-895a-0698065cd4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:36.819020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7223d23b-dcac-4d76-895a-0698065cd4bd None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:57:36.820159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7223d23b-dcac-4d76-895a-0698065cd4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:57:36.926010 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:57:37.080267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895abc3e-ff23-4132-ad61-1c2d4bb71b63 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:57:37.084334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-895abc3e-ff23-4132-ad61-1c2d4bb71b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:57:37.200321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895abc3e-ff23-4132-ad61-1c2d4bb71b63 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:57:37.206580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e2ab0-9edf-48e0-967c-fb3f09157c93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:57:37.207187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2ab0-9edf-48e0-967c-fb3f09157c93 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:37.207469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2ab0-9edf-48e0-967c-fb3f09157c93 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:37.207749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2ab0-9edf-48e0-967c-fb3f09157c93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:37.216022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e2ab0-9edf-48e0-967c-fb3f09157c93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:57:37.217643 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e2ab0-9edf-48e0-967c-fb3f09157c93 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:57:37.227691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 751/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:57:38.057816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c13c9-e8a3-4392-9c0b-8b0ec51a4a42 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:38.058378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c13c9-e8a3-4392-9c0b-8b0ec51a4a42 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:38.058378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c13c9-e8a3-4392-9c0b-8b0ec51a4a42 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:38.059732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:57:38.065143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e7522-fe1f-46e7-a057-a8328afebbae None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:38.065556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e7522-fe1f-46e7-a057-a8328afebbae None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:38.065699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e7522-fe1f-46e7-a057-a8328afebbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:38.086334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e28e7522-fe1f-46e7-a057-a8328afebbae None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:57:38.087554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e28e7522-fe1f-46e7-a057-a8328afebbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:57:38.095423 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:57:38.248609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c13c9-e8a3-4392-9c0b-8b0ec51a4a42 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:57:38.262924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c13c9-e8a3-4392-9c0b-8b0ec51a4a42 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:57:38.268739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c08d7-13d8-4621-8956-3c3b07a58fb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:57:38.269377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c08d7-13d8-4621-8956-3c3b07a58fb4 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:38.269661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c08d7-13d8-4621-8956-3c3b07a58fb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:38.269934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c08d7-13d8-4621-8956-3c3b07a58fb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:38.278144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c08d7-13d8-4621-8956-3c3b07a58fb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:57:38.279723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c08d7-13d8-4621-8956-3c3b07a58fb4 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:57:38.289679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 764/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:57:38.637276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27d3b6-0e20-4580-add4-a69ee4d769c3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:38.637563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27d3b6-0e20-4580-add4-a69ee4d769c3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:38.637840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27d3b6-0e20-4580-add4-a69ee4d769c3 None None] PATH_INFO: `/v3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:38.639120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 753/1517] 10.222.0.23 () {54 vars in 925 bytes} [Thu Aug 11 09:57:38 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:57:38.645596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f681695-6c65-4d6d-bc10-68009a0b8f9c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:38.645889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f681695-6c65-4d6d-bc10-68009a0b8f9c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:38.646173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f681695-6c65-4d6d-bc10-68009a0b8f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:38.704861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4f681695-6c65-4d6d-bc10-68009a0b8f9c None None] MFA Rules not processed for user `10ce125d46b545499de1fcf7072963e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:57:38.706094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4f681695-6c65-4d6d-bc10-68009a0b8f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:57:38.723891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:57:38.777893 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 765/1518] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Aug 11 09:57:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:57:47.802486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab185b-6a46-4aec-b358-26f2da66ffac None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:47.803017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab185b-6a46-4aec-b358-26f2da66ffac None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:47.803017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab185b-6a46-4aec-b358-26f2da66ffac None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:47.804318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 754/1519] 10.222.0.17 () {54 vars in 961 bytes} [Thu Aug 11 09:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:57:47.809761 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9504d43-bffb-4817-b37c-18abeee157c3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:47.810180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9504d43-bffb-4817-b37c-18abeee157c3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:47.810311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9504d43-bffb-4817-b37c-18abeee157c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:47.830511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f9504d43-bffb-4817-b37c-18abeee157c3 None None] MFA Rules not processed for user `10ce125d46b545499de1fcf7072963e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:57:47.831723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f9504d43-bffb-4817-b37c-18abeee157c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:57:47.840473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 766/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Aug 11 09:57:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:57:47.848154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab185b-6a46-4aec-b358-26f2da66ffac None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:57:47.861337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab185b-6a46-4aec-b358-26f2da66ffac None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:57:47.866287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479e3e5-b51c-4c1b-a9e0-900de582032f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:57:47.866872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479e3e5-b51c-4c1b-a9e0-900de582032f service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:47.867149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479e3e5-b51c-4c1b-a9e0-900de582032f service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:47.867420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479e3e5-b51c-4c1b-a9e0-900de582032f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:47.875984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479e3e5-b51c-4c1b-a9e0-900de582032f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:57:47.877617 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479e3e5-b51c-4c1b-a9e0-900de582032f service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:57:47.886970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 755/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:57:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 09:57:49.529658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d1a4e-6334-4e41-8dde-a2161cdf1d04 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:49.530191 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d1a4e-6334-4e41-8dde-a2161cdf1d04 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:49.530191 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d1a4e-6334-4e41-8dde-a2161cdf1d04 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:49.532177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:57:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:57:49.537071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e288a5-1f58-434d-a25d-3b09fbaa55f9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:49.537513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e288a5-1f58-434d-a25d-3b09fbaa55f9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:49.537630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e288a5-1f58-434d-a25d-3b09fbaa55f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:49.557319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a3e288a5-1f58-434d-a25d-3b09fbaa55f9 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:57:49.558519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a3e288a5-1f58-434d-a25d-3b09fbaa55f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:57:49.566940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:57:49.722938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d1a4e-6334-4e41-8dde-a2161cdf1d04 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:57:49.738178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d1a4e-6334-4e41-8dde-a2161cdf1d04 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:57:49.743849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ee9f4-5bf0-49a2-9daf-66212acda43f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:57:49.744444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ee9f4-5bf0-49a2-9daf-66212acda43f service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:49.744762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ee9f4-5bf0-49a2-9daf-66212acda43f service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:49.745132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ee9f4-5bf0-49a2-9daf-66212acda43f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:49.753572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ee9f4-5bf0-49a2-9daf-66212acda43f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:57:49.755478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ee9f4-5bf0-49a2-9daf-66212acda43f service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:57:49.765913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 768/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:57:50.577061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b799c-8c6e-4930-9a75-80085e6ac809 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:50.577596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b799c-8c6e-4930-9a75-80085e6ac809 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:50.577753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b799c-8c6e-4930-9a75-80085e6ac809 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:50.579313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 09:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:57:50.585114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901c3b2-c6e6-4a3e-a03b-ae65e8e89323 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:50.585551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901c3b2-c6e6-4a3e-a03b-ae65e8e89323 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:50.585686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901c3b2-c6e6-4a3e-a03b-ae65e8e89323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:50.605584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a901c3b2-c6e6-4a3e-a03b-ae65e8e89323 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:57:50.606786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a901c3b2-c6e6-4a3e-a03b-ae65e8e89323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:57:50.615071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 09:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:57:50.773197 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b799c-8c6e-4930-9a75-80085e6ac809 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:57:50.788209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b799c-8c6e-4930-9a75-80085e6ac809 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:57:50.794051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642ebd-2b31-41cf-9395-2e7c4104268f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:57:50.794652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642ebd-2b31-41cf-9395-2e7c4104268f service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:50.794930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642ebd-2b31-41cf-9395-2e7c4104268f service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:50.795207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642ebd-2b31-41cf-9395-2e7c4104268f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:50.803665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42642ebd-2b31-41cf-9395-2e7c4104268f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:57:50.807154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42642ebd-2b31-41cf-9395-2e7c4104268f service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:57:50.815368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 758/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:57:54.731254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e4f7f-42d4-4c1c-bcb5-2ef35e774fb8 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:54.731834 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e4f7f-42d4-4c1c-bcb5-2ef35e774fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:54.731834 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e4f7f-42d4-4c1c-bcb5-2ef35e774fb8 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:54.733132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:57:54.738283 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1231f3a-0900-4e01-b7d4-bc0cb801ecc7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:54.738723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1231f3a-0900-4e01-b7d4-bc0cb801ecc7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:54.738831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1231f3a-0900-4e01-b7d4-bc0cb801ecc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:54.759276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b1231f3a-0900-4e01-b7d4-bc0cb801ecc7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:57:54.760480 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b1231f3a-0900-4e01-b7d4-bc0cb801ecc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:57:54.768967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:57:54.841919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e4f7f-42d4-4c1c-bcb5-2ef35e774fb8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:57:54.843648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-395e4f7f-42d4-4c1c-bcb5-2ef35e774fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:57:54.921911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e4f7f-42d4-4c1c-bcb5-2ef35e774fb8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:57:54.925659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658740f-138d-4ce5-bd41-5e2339dc511f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:57:54.926260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658740f-138d-4ce5-bd41-5e2339dc511f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:54.926540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658740f-138d-4ce5-bd41-5e2339dc511f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:54.926810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658740f-138d-4ce5-bd41-5e2339dc511f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:54.935607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658740f-138d-4ce5-bd41-5e2339dc511f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:57:54.938152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658740f-138d-4ce5-bd41-5e2339dc511f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:57:54.947787 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 771/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:57:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:57:56.931947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5402747-0349-41f6-9004-7781a56c4a11 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:56.932490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5402747-0349-41f6-9004-7781a56c4a11 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:56.932615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5402747-0349-41f6-9004-7781a56c4a11 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:56.933964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:57:56.939493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b73f4-bcd4-446a-8316-548dbb1447d1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:56.939946 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b73f4-bcd4-446a-8316-548dbb1447d1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:56.940060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b73f4-bcd4-446a-8316-548dbb1447d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:56.960205 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d57b73f4-bcd4-446a-8316-548dbb1447d1 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:57:56.961694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d57b73f4-bcd4-446a-8316-548dbb1447d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:57:56.970103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:57:57.005421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5402747-0349-41f6-9004-7781a56c4a11 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:57:57.016531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5402747-0349-41f6-9004-7781a56c4a11 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:57:57.020028 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdcf15-e257-4b2d-ba61-3e9e086694c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:57:57.020620 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdcf15-e257-4b2d-ba61-3e9e086694c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:57:57.020895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdcf15-e257-4b2d-ba61-3e9e086694c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:57:57.021192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdcf15-e257-4b2d-ba61-3e9e086694c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:57:57.029883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdcf15-e257-4b2d-ba61-3e9e086694c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:57:57.031321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdcf15-e257-4b2d-ba61-3e9e086694c5 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:57:57.040483 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 761/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:58:00.529838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e2e5e-3e64-4ae6-b011-11bbceb2779f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:58:00.530931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e2e5e-3e64-4ae6-b011-11bbceb2779f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:58:00.531039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e2e5e-3e64-4ae6-b011-11bbceb2779f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:58:00.532320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:58:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:58:00.537540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c74f5-9c5a-40f1-9d42-2754bc49e334 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:58:00.537949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c74f5-9c5a-40f1-9d42-2754bc49e334 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:58:00.538090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c74f5-9c5a-40f1-9d42-2754bc49e334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:58:00.545619 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=81 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:58:00.558040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-464c74f5-9c5a-40f1-9d42-2754bc49e334 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:58:00.559364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-464c74f5-9c5a-40f1-9d42-2754bc49e334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:58:00.567241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:58:00.872505 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e2e5e-3e64-4ae6-b011-11bbceb2779f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:58:00.879142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cc4e2e5e-3e64-4ae6-b011-11bbceb2779f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:58:00.938997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:58:00.953117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e2e5e-3e64-4ae6-b011-11bbceb2779f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:58:00.956906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b0526-d40d-4465-9d9a-6542539108c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:58:00.957345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b0526-d40d-4465-9d9a-6542539108c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:58:00.957618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b0526-d40d-4465-9d9a-6542539108c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:58:00.957884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b0526-d40d-4465-9d9a-6542539108c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:58:00.966855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b0526-d40d-4465-9d9a-6542539108c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:58:00.968891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b0526-d40d-4465-9d9a-6542539108c5 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:58:00.979000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 774/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:58:03.016778 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4da74-5b09-4604-bbdc-cbbc917dc093 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:58:03.017371 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4da74-5b09-4604-bbdc-cbbc917dc093 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:58:03.017371 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4da74-5b09-4604-bbdc-cbbc917dc093 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:58:03.018705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 09:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:58:03.023887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8c8e1-16ef-4c3f-93e5-c48d89dbaffe None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:58:03.024303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8c8e1-16ef-4c3f-93e5-c48d89dbaffe None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:58:03.024412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc8c8e1-16ef-4c3f-93e5-c48d89dbaffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:58:03.043419 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1cc8c8e1-16ef-4c3f-93e5-c48d89dbaffe None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:58:03.044608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1cc8c8e1-16ef-4c3f-93e5-c48d89dbaffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:58:03.052343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 09:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:58:03.354992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4da74-5b09-4604-bbdc-cbbc917dc093 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:58:03.356715 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b1e4da74-5b09-4604-bbdc-cbbc917dc093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:58:03.369561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4da74-5b09-4604-bbdc-cbbc917dc093 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:58:03.373003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6e9ae-a580-4205-a140-79551b5eb5e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:58:03.373615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e9ae-a580-4205-a140-79551b5eb5e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:58:03.373964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e9ae-a580-4205-a140-79551b5eb5e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:58:03.374256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6e9ae-a580-4205-a140-79551b5eb5e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:58:03.383140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6e9ae-a580-4205-a140-79551b5eb5e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:58:03.384574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6e9ae-a580-4205-a140-79551b5eb5e3 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:58:03.394362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 764/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:58:13.038756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=11 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:58:13.389838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=8 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:25.165064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1e58c-8930-43f4-97c6-99cdf83e1ecf None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.180805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1e58c-8930-43f4-97c6-99cdf83e1ecf None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.180805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1e58c-8930-43f4-97c6-99cdf83e1ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.209916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4cd1e58c-8930-43f4-97c6-99cdf83e1ecf None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.211541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4cd1e58c-8930-43f4-97c6-99cdf83e1ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.224243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 776/1540] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.233215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd634f-0be2-4b15-a024-8c173b6ee25f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.233844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd634f-0be2-4b15-a024-8c173b6ee25f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.233844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd634f-0be2-4b15-a024-8c173b6ee25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.269292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1ddd634f-0be2-4b15-a024-8c173b6ee25f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.270538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1ddd634f-0be2-4b15-a024-8c173b6ee25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.283690 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 765/1541] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.288376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1e58c-8930-43f4-97c6-99cdf83e1ecf None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:25.303208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1e58c-8930-43f4-97c6-99cdf83e1ecf None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:25.312023 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4b3b7-5f29-47fb-9e27-900a7087407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:25.313086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4b3b7-5f29-47fb-9e27-900a7087407d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.313604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4b3b7-5f29-47fb-9e27-900a7087407d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.314886 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4b3b7-5f29-47fb-9e27-900a7087407d admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.315820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4b3b7-5f29-47fb-9e27-900a7087407d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:25.316368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4b3b7-5f29-47fb-9e27-900a7087407d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:25.318299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4b3b7-5f29-47fb-9e27-900a7087407d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:25.342538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 777/1542] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:25.351093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd634f-0be2-4b15-a024-8c173b6ee25f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:25.366119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd634f-0be2-4b15-a024-8c173b6ee25f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:25.382168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71337c7-ed45-436c-a9f8-3211fbb58b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:25.383107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71337c7-ed45-436c-a9f8-3211fbb58b09 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.383952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71337c7-ed45-436c-a9f8-3211fbb58b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.384391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71337c7-ed45-436c-a9f8-3211fbb58b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.385244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71337c7-ed45-436c-a9f8-3211fbb58b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:25.387614 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71337c7-ed45-436c-a9f8-3211fbb58b09 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:25.398404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c6201-b358-4a1a-94aa-9d7e7c1d85bf None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.399119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c6201-b358-4a1a-94aa-9d7e7c1d85bf None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.399119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c6201-b358-4a1a-94aa-9d7e7c1d85bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.439283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2a9c6201-b358-4a1a-94aa-9d7e7c1d85bf None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.439283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2a9c6201-b358-4a1a-94aa-9d7e7c1d85bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.448082 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 778/1543] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.458409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dd5e1-6b05-4bec-8f70-12285b490159 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.458840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dd5e1-6b05-4bec-8f70-12285b490159 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.459227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dd5e1-6b05-4bec-8f70-12285b490159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.492746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-466dd5e1-6b05-4bec-8f70-12285b490159 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.494085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-466dd5e1-6b05-4bec-8f70-12285b490159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.504655 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 779/1544] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.511302 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466dd5e1-6b05-4bec-8f70-12285b490159 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:25.525686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466dd5e1-6b05-4bec-8f70-12285b490159 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:25.533051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fd4af-65b7-4387-bee3-c64c189c96e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:25.533678 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fd4af-65b7-4387-bee3-c64c189c96e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.533961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fd4af-65b7-4387-bee3-c64c189c96e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.534239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fd4af-65b7-4387-bee3-c64c189c96e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.534893 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fd4af-65b7-4387-bee3-c64c189c96e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:25.535197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fd4af-65b7-4387-bee3-c64c189c96e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:25.536792 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fd4af-65b7-4387-bee3-c64c189c96e3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:25.555201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 780/1545] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 09:59:25.556485 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556ed09-5248-4a59-8cbb-4ac412a27248 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.556765 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556ed09-5248-4a59-8cbb-4ac412a27248 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.557055 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556ed09-5248-4a59-8cbb-4ac412a27248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.558337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 766/1546] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/projects => generated 381 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:25.559968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fabc86-6462-47c7-843d-6b84c4ad6e3a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.560489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fabc86-6462-47c7-843d-6b84c4ad6e3a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.560988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fabc86-6462-47c7-843d-6b84c4ad6e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.596672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-8556ed09-5248-4a59-8cbb-4ac412a27248 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.599558 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8556ed09-5248-4a59-8cbb-4ac412a27248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.603616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-26fabc86-6462-47c7-843d-6b84c4ad6e3a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.606015 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-26fabc86-6462-47c7-843d-6b84c4ad6e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.676801 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 781/1547] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.677689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556ed09-5248-4a59-8cbb-4ac412a27248 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:25.686816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556ed09-5248-4a59-8cbb-4ac412a27248 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:25.689436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 767/1548] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.690209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fabc86-6462-47c7-843d-6b84c4ad6e3a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:25.696173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7380f53-582b-4e61-aeac-24ffc1415761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:25.696984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7380f53-582b-4e61-aeac-24ffc1415761 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.697144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7380f53-582b-4e61-aeac-24ffc1415761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.697376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7380f53-582b-4e61-aeac-24ffc1415761 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.698006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7380f53-582b-4e61-aeac-24ffc1415761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:25.699763 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fabc86-6462-47c7-843d-6b84c4ad6e3a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:25.699921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7380f53-582b-4e61-aeac-24ffc1415761 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:25.705077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89bc3d-a90f-47fb-b3bc-5ee73a056f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:25.705806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89bc3d-a90f-47fb-b3bc-5ee73a056f4b admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.706089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89bc3d-a90f-47fb-b3bc-5ee73a056f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.706362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89bc3d-a90f-47fb-b3bc-5ee73a056f4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.706987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89bc3d-a90f-47fb-b3bc-5ee73a056f4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:25.708874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89bc3d-a90f-47fb-b3bc-5ee73a056f4b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:25.790823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 782/1549] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:25.792284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a9170-dbca-4fa3-8639-a90d3882e05e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.792573 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a9170-dbca-4fa3-8639-a90d3882e05e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.792849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a9170-dbca-4fa3-8639-a90d3882e05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.809277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 768/1550] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/projects => generated 393 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:25.810549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f6b7a-a6a9-48d8-8c3c-e1489aec326b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.812885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f6b7a-a6a9-48d8-8c3c-e1489aec326b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.812885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f6b7a-a6a9-48d8-8c3c-e1489aec326b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.813092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b69a9170-dbca-4fa3-8639-a90d3882e05e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.814239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b69a9170-dbca-4fa3-8639-a90d3882e05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.830564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c17f6b7a-a6a9-48d8-8c3c-e1489aec326b None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.831762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c17f6b7a-a6a9-48d8-8c3c-e1489aec326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.875728 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 783/1551] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.878510 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abdaa9-5272-4043-9f48-ad5a343386d2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.878829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abdaa9-5272-4043-9f48-ad5a343386d2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.879110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abdaa9-5272-4043-9f48-ad5a343386d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.895099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 769/1552] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.896354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27661691-5b67-42fb-90b3-644b8d6d873a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.896635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27661691-5b67-42fb-90b3-644b8d6d873a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.896905 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27661691-5b67-42fb-90b3-644b8d6d873a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.906167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-39abdaa9-5272-4043-9f48-ad5a343386d2 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.908071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-39abdaa9-5272-4043-9f48-ad5a343386d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.916100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 784/1553] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.916311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-27661691-5b67-42fb-90b3-644b8d6d873a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.917399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668677ed-8252-44e5-9069-d95ac0e6bc1a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.917525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-27661691-5b67-42fb-90b3-644b8d6d873a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.917688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668677ed-8252-44e5-9069-d95ac0e6bc1a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.917956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668677ed-8252-44e5-9069-d95ac0e6bc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.929932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 770/1554] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.929932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aed22-23b9-4f1c-be95-60f594f1c310 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.929932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aed22-23b9-4f1c-be95-60f594f1c310 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.929932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aed22-23b9-4f1c-be95-60f594f1c310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.941226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-668677ed-8252-44e5-9069-d95ac0e6bc1a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.942375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-668677ed-8252-44e5-9069-d95ac0e6bc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.947818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cc1aed22-23b9-4f1c-be95-60f594f1c310 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:25.949036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cc1aed22-23b9-4f1c-be95-60f594f1c310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:25.950301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 785/1555] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.951177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668677ed-8252-44e5-9069-d95ac0e6bc1a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:25.959307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 771/1556] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:25.960080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1aed22-23b9-4f1c-be95-60f594f1c310 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:25.961299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668677ed-8252-44e5-9069-d95ac0e6bc1a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:25.966261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f58bf-9cff-4b34-b94e-6555b3fa579a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:25.966851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f58bf-9cff-4b34-b94e-6555b3fa579a admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.967130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f58bf-9cff-4b34-b94e-6555b3fa579a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.967403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f58bf-9cff-4b34-b94e-6555b3fa579a admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.968065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f58bf-9cff-4b34-b94e-6555b3fa579a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:25.969662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681f58bf-9cff-4b34-b94e-6555b3fa579a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:25.970561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1aed22-23b9-4f1c-be95-60f594f1c310 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:25.975504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5510d-c8b7-43b7-a2d3-ebfee98ce9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:25.976091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5510d-c8b7-43b7-a2d3-ebfee98ce9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.976368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5510d-c8b7-43b7-a2d3-ebfee98ce9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.976640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5510d-c8b7-43b7-a2d3-ebfee98ce9d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:25.977334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5510d-c8b7-43b7-a2d3-ebfee98ce9d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:25.978326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-6bc5510d-c8b7-43b7-a2d3-ebfee98ce9d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:59:25.978326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 11 09:59:25.978326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: Aug 11 09:59:25.979510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5510d-c8b7-43b7-a2d3-ebfee98ce9d0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:25.982878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 786/1557] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:25.982878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07712c4-f3a6-47e1-a531-58d30464e472 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:25.982878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07712c4-f3a6-47e1-a531-58d30464e472 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:25.982878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07712c4-f3a6-47e1-a531-58d30464e472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.000839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c07712c4-f3a6-47e1-a531-58d30464e472 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:26.002595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c07712c4-f3a6-47e1-a531-58d30464e472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:26.010525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 787/1558] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:26.011673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b703ff-c12d-42d3-a2e0-346d24e9c1a3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.011955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b703ff-c12d-42d3-a2e0-346d24e9c1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.012225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b703ff-c12d-42d3-a2e0-346d24e9c1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.025329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:26.030982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b3b703ff-c12d-42d3-a2e0-346d24e9c1a3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:26.032161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b3b703ff-c12d-42d3-a2e0-346d24e9c1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:26.041298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 788/1559] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:26.042551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377216b-4ed2-413b-91c9-1b7e4dab2208 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.042950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377216b-4ed2-413b-91c9-1b7e4dab2208 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.043486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377216b-4ed2-413b-91c9-1b7e4dab2208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.062670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-8377216b-4ed2-413b-91c9-1b7e4dab2208 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:26.063917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8377216b-4ed2-413b-91c9-1b7e4dab2208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:26.072154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 789/1560] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:26.081322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5d690-2565-4968-b097-d2dc163b1f32 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.081322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5d690-2565-4968-b097-d2dc163b1f32 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.081322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5d690-2565-4968-b097-d2dc163b1f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.081624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 772/1561] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:25 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:26.081624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5510d-c8b7-43b7-a2d3-ebfee98ce9d0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.088293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5510d-c8b7-43b7-a2d3-ebfee98ce9d0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.092698 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4da5d690-2565-4968-b097-d2dc163b1f32 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:26.093003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b30e8-9b40-4618-a05d-dc29bda63398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.093635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b30e8-9b40-4618-a05d-dc29bda63398 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.093912 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4da5d690-2565-4968-b097-d2dc163b1f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:26.094025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b30e8-9b40-4618-a05d-dc29bda63398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.094185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b30e8-9b40-4618-a05d-dc29bda63398 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.095438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b30e8-9b40-4618-a05d-dc29bda63398 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:26.095691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b30e8-9b40-4618-a05d-dc29bda63398 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.096800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-6a0b30e8-9b40-4618-a05d-dc29bda63398 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:59:26.096800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 11 09:59:26.096800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: Aug 11 09:59:26.097804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b30e8-9b40-4618-a05d-dc29bda63398 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.103599 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 790/1562] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:26.105178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:26.116135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5d690-2565-4968-b097-d2dc163b1f32 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.116263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 773/1563] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:26.116263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b30e8-9b40-4618-a05d-dc29bda63398 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.122329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5d690-2565-4968-b097-d2dc163b1f32 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.122686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b30e8-9b40-4618-a05d-dc29bda63398 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.127525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7d2f9-bffb-4cec-b6e3-00bce3180565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.127844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae728d6a-ad7c-49dc-85b0-43831c215288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.128261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7d2f9-bffb-4cec-b6e3-00bce3180565 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.128508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae728d6a-ad7c-49dc-85b0-43831c215288 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.128630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7d2f9-bffb-4cec-b6e3-00bce3180565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.128803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae728d6a-ad7c-49dc-85b0-43831c215288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.128916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7d2f9-bffb-4cec-b6e3-00bce3180565 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.129108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae728d6a-ad7c-49dc-85b0-43831c215288 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.129496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7d2f9-bffb-4cec-b6e3-00bce3180565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.129894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae728d6a-ad7c-49dc-85b0-43831c215288 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:26.130202 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae728d6a-ad7c-49dc-85b0-43831c215288 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.131075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7d2f9-bffb-4cec-b6e3-00bce3180565 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.131829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae728d6a-ad7c-49dc-85b0-43831c215288 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.141255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 774/1564] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:26.142543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7d2f9-bffb-4cec-b6e3-00bce3180565 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.150909 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 791/1565] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:26.150909 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae728d6a-ad7c-49dc-85b0-43831c215288 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.152547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7d2f9-bffb-4cec-b6e3-00bce3180565 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.156073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae728d6a-ad7c-49dc-85b0-43831c215288 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.159045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e756fca-933a-4376-bbc7-cde5cbd4049b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.159681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e756fca-933a-4376-bbc7-cde5cbd4049b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.159965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e756fca-933a-4376-bbc7-cde5cbd4049b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.160244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e756fca-933a-4376-bbc7-cde5cbd4049b admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.160891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e756fca-933a-4376-bbc7-cde5cbd4049b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:26.161227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e756fca-933a-4376-bbc7-cde5cbd4049b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.161426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276a299-e504-437e-a0fb-003397985037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.162059 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276a299-e504-437e-a0fb-003397985037 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.162344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276a299-e504-437e-a0fb-003397985037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.162613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276a299-e504-437e-a0fb-003397985037 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.162841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e756fca-933a-4376-bbc7-cde5cbd4049b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.163250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276a299-e504-437e-a0fb-003397985037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:26.163549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276a299-e504-437e-a0fb-003397985037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.165553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276a299-e504-437e-a0fb-003397985037 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.183729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 775/1566] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:26.183729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e756fca-933a-4376-bbc7-cde5cbd4049b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.184029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 792/1567] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:26.184029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276a299-e504-437e-a0fb-003397985037 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.192446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276a299-e504-437e-a0fb-003397985037 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.195466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e756fca-933a-4376-bbc7-cde5cbd4049b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.198199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb4dda-0af6-49f0-b256-de2d7016fc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.198999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb4dda-0af6-49f0-b256-de2d7016fc53 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.199399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb4dda-0af6-49f0-b256-de2d7016fc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.199665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb4dda-0af6-49f0-b256-de2d7016fc53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.200149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa58a8-1b5a-404e-991a-bb448bb7c062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.200293 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb4dda-0af6-49f0-b256-de2d7016fc53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:26.200592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb4dda-0af6-49f0-b256-de2d7016fc53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.200750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa58a8-1b5a-404e-991a-bb448bb7c062 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.201052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa58a8-1b5a-404e-991a-bb448bb7c062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.201334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa58a8-1b5a-404e-991a-bb448bb7c062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.201980 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa58a8-1b5a-404e-991a-bb448bb7c062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.202270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb4dda-0af6-49f0-b256-de2d7016fc53 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.203545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa58a8-1b5a-404e-991a-bb448bb7c062 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.213932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 776/1568] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:26.214739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa58a8-1b5a-404e-991a-bb448bb7c062 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.217258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 793/1569] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:26.217258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb4dda-0af6-49f0-b256-de2d7016fc53 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.226879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb4dda-0af6-49f0-b256-de2d7016fc53 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.227861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa58a8-1b5a-404e-991a-bb448bb7c062 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.231484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e8f1f-120c-4e24-a535-26327e5281aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.232101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e8f1f-120c-4e24-a535-26327e5281aa admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.232383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e8f1f-120c-4e24-a535-26327e5281aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.232508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b90eaf-a1ce-4e8b-9827-ce554caf709e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.232633 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e8f1f-120c-4e24-a535-26327e5281aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.233208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b90eaf-a1ce-4e8b-9827-ce554caf709e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.233328 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e8f1f-120c-4e24-a535-26327e5281aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.233493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b90eaf-a1ce-4e8b-9827-ce554caf709e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.233945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b90eaf-a1ce-4e8b-9827-ce554caf709e admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.234263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-e81e8f1f-120c-4e24-a535-26327e5281aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:59:26.234263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 11 09:59:26.234263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: Aug 11 09:59:26.234502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b90eaf-a1ce-4e8b-9827-ce554caf709e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:26.234688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b90eaf-a1ce-4e8b-9827-ce554caf709e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.235428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81e8f1f-120c-4e24-a535-26327e5281aa admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.237206 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b90eaf-a1ce-4e8b-9827-ce554caf709e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.249423 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 777/1570] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:26.250467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b90eaf-a1ce-4e8b-9827-ce554caf709e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.260310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b90eaf-a1ce-4e8b-9827-ce554caf709e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.264885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ab8e6-e669-4d24-b955-31e3efdc0fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.265548 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ab8e6-e669-4d24-b955-31e3efdc0fc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.268268 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ab8e6-e669-4d24-b955-31e3efdc0fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.268268 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ab8e6-e669-4d24-b955-31e3efdc0fc8 admin admin] PATH_INFO: `/v3/projects/878ea5b76a9d4139bd0a3461d12b3ab0/users/7cb12d703dc148688958302e869eb71f/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.280859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ab8e6-e669-4d24-b955-31e3efdc0fc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=878ea5b76a9d4139bd0a3461d12b3ab0, user_id=7cb12d703dc148688958302e869eb71f, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.284139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ab8e6-e669-4d24-b955-31e3efdc0fc8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.326626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 794/1571] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:26.326626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e8f1f-120c-4e24-a535-26327e5281aa admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.350952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 778/1572] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:26 2022] PUT /identity/v3/projects/878ea5b76a9d4139bd0a3461d12b3ab0/users/7cb12d703dc148688958302e869eb71f/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:26.351827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ab8e6-e669-4d24-b955-31e3efdc0fc8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.386998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81e8f1f-120c-4e24-a535-26327e5281aa admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.422370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ab8e6-e669-4d24-b955-31e3efdc0fc8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.427612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d99766-bce9-45c5-aa64-09b2b79b8ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.428216 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d99766-bce9-45c5-aa64-09b2b79b8ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.428493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d99766-bce9-45c5-aa64-09b2b79b8ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.428760 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d99766-bce9-45c5-aa64-09b2b79b8ec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.429427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d99766-bce9-45c5-aa64-09b2b79b8ec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.431299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d99766-bce9-45c5-aa64-09b2b79b8ec4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.442680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb380ae2-c7b3-4465-960a-dfe730ed6899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.443248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb380ae2-c7b3-4465-960a-dfe730ed6899 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.443522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb380ae2-c7b3-4465-960a-dfe730ed6899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.443791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb380ae2-c7b3-4465-960a-dfe730ed6899 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.444413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb380ae2-c7b3-4465-960a-dfe730ed6899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.446340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb380ae2-c7b3-4465-960a-dfe730ed6899 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.540801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 779/1573] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:26.541642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d99766-bce9-45c5-aa64-09b2b79b8ec4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.552331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 795/1574] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/projects => generated 405 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:26.553508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb380ae2-c7b3-4465-960a-dfe730ed6899 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.572871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:26.611058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d99766-bce9-45c5-aa64-09b2b79b8ec4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.615989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b47510-0501-4b40-88c2-08f59edac02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.616316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb380ae2-c7b3-4465-960a-dfe730ed6899 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.616764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b47510-0501-4b40-88c2-08f59edac02b admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.618306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b47510-0501-4b40-88c2-08f59edac02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.618604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b47510-0501-4b40-88c2-08f59edac02b admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.619252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b47510-0501-4b40-88c2-08f59edac02b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.621288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b47510-0501-4b40-88c2-08f59edac02b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.622740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759fc4f-de0f-4683-a080-8278a4d22878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.623347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759fc4f-de0f-4683-a080-8278a4d22878 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.623614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759fc4f-de0f-4683-a080-8278a4d22878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.623884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759fc4f-de0f-4683-a080-8278a4d22878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.624563 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759fc4f-de0f-4683-a080-8278a4d22878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.626312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759fc4f-de0f-4683-a080-8278a4d22878 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.637612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 796/1575] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:26.638538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759fc4f-de0f-4683-a080-8278a4d22878 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.641403 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:26.648749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759fc4f-de0f-4683-a080-8278a4d22878 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.653363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7fff3-ee3e-40ad-82e7-003a6c6d0891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.653936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7fff3-ee3e-40ad-82e7-003a6c6d0891 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.654211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7fff3-ee3e-40ad-82e7-003a6c6d0891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.654477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7fff3-ee3e-40ad-82e7-003a6c6d0891 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.655085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7fff3-ee3e-40ad-82e7-003a6c6d0891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.656987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7fff3-ee3e-40ad-82e7-003a6c6d0891 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.705260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 780/1576] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/projects => generated 405 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:26.706297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b47510-0501-4b40-88c2-08f59edac02b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.757102 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 797/1577] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:26.758196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7fff3-ee3e-40ad-82e7-003a6c6d0891 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.766367 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b47510-0501-4b40-88c2-08f59edac02b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.820334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7fff3-ee3e-40ad-82e7-003a6c6d0891 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.821238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166a8f6-1a0e-444a-8af3-0c60805e173f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.821852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166a8f6-1a0e-444a-8af3-0c60805e173f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.822128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166a8f6-1a0e-444a-8af3-0c60805e173f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.822395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166a8f6-1a0e-444a-8af3-0c60805e173f admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.823014 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166a8f6-1a0e-444a-8af3-0c60805e173f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.824885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166a8f6-1a0e-444a-8af3-0c60805e173f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.825136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe76f0-b788-403c-b87a-c3e123dc2551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.825740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe76f0-b788-403c-b87a-c3e123dc2551 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.826019 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe76f0-b788-403c-b87a-c3e123dc2551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.826292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe76f0-b788-403c-b87a-c3e123dc2551 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.826916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe76f0-b788-403c-b87a-c3e123dc2551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.828813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe76f0-b788-403c-b87a-c3e123dc2551 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:26.911243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 781/1578] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:26.912644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4779a-adfd-4ce7-8960-82e8aed87d09 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.912933 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4779a-adfd-4ce7-8960-82e8aed87d09 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.913244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4779a-adfd-4ce7-8960-82e8aed87d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.921919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 798/1579] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:26.922616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe76f0-b788-403c-b87a-c3e123dc2551 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:26.932170 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe76f0-b788-403c-b87a-c3e123dc2551 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:26.947001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-eaf4779a-adfd-4ce7-8960-82e8aed87d09 None None] MFA Rules not processed for user `7cb12d703dc148688958302e869eb71f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:26.948267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-eaf4779a-adfd-4ce7-8960-82e8aed87d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:26.990264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a44576-6ace-4eb0-b05f-dbf60c12e062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:26.990790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a44576-6ace-4eb0-b05f-dbf60c12e062 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:26.991072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a44576-6ace-4eb0-b05f-dbf60c12e062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:26.991358 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a44576-6ace-4eb0-b05f-dbf60c12e062 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:26.991981 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a44576-6ace-4eb0-b05f-dbf60c12e062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:26.993964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a44576-6ace-4eb0-b05f-dbf60c12e062 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.013273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 782/1580] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:27.017989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4779a-adfd-4ce7-8960-82e8aed87d09 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.024509 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4779a-adfd-4ce7-8960-82e8aed87d09 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.029041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d456ca3-075c-48d1-9a43-fdf7626fa5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.029609 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d456ca3-075c-48d1-9a43-fdf7626fa5a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.029885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d456ca3-075c-48d1-9a43-fdf7626fa5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.030158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d456ca3-075c-48d1-9a43-fdf7626fa5a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.030774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d456ca3-075c-48d1-9a43-fdf7626fa5a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.032696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d456ca3-075c-48d1-9a43-fdf7626fa5a3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.081993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 799/1581] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:26 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:27.082801 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a44576-6ace-4eb0-b05f-dbf60c12e062 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.092469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a44576-6ace-4eb0-b05f-dbf60c12e062 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.094579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:27.097357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2df9c3-69d9-4ad1-80cb-c391b76f7638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.098006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2df9c3-69d9-4ad1-80cb-c391b76f7638 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.098276 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2df9c3-69d9-4ad1-80cb-c391b76f7638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.098560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2df9c3-69d9-4ad1-80cb-c391b76f7638 admin admin] PATH_INFO: `/v3/projects/75fb7bc39c364a5fb80532cbd3b2f654/users/c62f05ea732142c1af1c09dd23e7b9a3/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.100192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 783/1582] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:27 2022] POST /identity/v3/users => generated 303 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:27.100962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d456ca3-075c-48d1-9a43-fdf7626fa5a3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.111129 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d456ca3-075c-48d1-9a43-fdf7626fa5a3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.115833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:27.115965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d6790-13dd-4170-805a-b1aa63841ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.116155 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d6790-13dd-4170-805a-b1aa63841ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.116444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d6790-13dd-4170-805a-b1aa63841ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.116719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d6790-13dd-4170-805a-b1aa63841ee1 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.117383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d6790-13dd-4170-805a-b1aa63841ee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.117955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2df9c3-69d9-4ad1-80cb-c391b76f7638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75fb7bc39c364a5fb80532cbd3b2f654, user_id=c62f05ea732142c1af1c09dd23e7b9a3, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.118994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-cc2df9c3-69d9-4ad1-80cb-c391b76f7638 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:59:27.118994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 11 09:59:27.118994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: Aug 11 09:59:27.119296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d6790-13dd-4170-805a-b1aa63841ee1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.121025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2df9c3-69d9-4ad1-80cb-c391b76f7638 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.213591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 800/1583] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:27 2022] PUT /identity/v3/projects/75fb7bc39c364a5fb80532cbd3b2f654/users/c62f05ea732142c1af1c09dd23e7b9a3/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:27.216973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2df9c3-69d9-4ad1-80cb-c391b76f7638 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.245437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 784/1584] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:27 2022] POST /identity/v3/users => generated 303 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:27.245437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d6790-13dd-4170-805a-b1aa63841ee1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.253769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d6790-13dd-4170-805a-b1aa63841ee1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.276836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2df9c3-69d9-4ad1-80cb-c391b76f7638 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.281734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf6fb7-42a5-41bf-8648-e80884d37f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.282466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf6fb7-42a5-41bf-8648-e80884d37f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.282889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf6fb7-42a5-41bf-8648-e80884d37f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.283343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf6fb7-42a5-41bf-8648-e80884d37f42 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.283972 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf6fb7-42a5-41bf-8648-e80884d37f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.286043 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf6fb7-42a5-41bf-8648-e80884d37f42 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.309063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752910b-1dba-4a83-bcb3-95750d407d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.309892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752910b-1dba-4a83-bcb3-95750d407d7c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.310365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752910b-1dba-4a83-bcb3-95750d407d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.310683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752910b-1dba-4a83-bcb3-95750d407d7c admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.311750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752910b-1dba-4a83-bcb3-95750d407d7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.314501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752910b-1dba-4a83-bcb3-95750d407d7c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.384317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 801/1585] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:27 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:27.385178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf6fb7-42a5-41bf-8648-e80884d37f42 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.394658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf6fb7-42a5-41bf-8648-e80884d37f42 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.399248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97d073-2349-4e22-a459-e550e211afa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.399820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97d073-2349-4e22-a459-e550e211afa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.400482 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97d073-2349-4e22-a459-e550e211afa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.401062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97d073-2349-4e22-a459-e550e211afa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.404928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97d073-2349-4e22-a459-e550e211afa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.404928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97d073-2349-4e22-a459-e550e211afa6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.408313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 785/1586] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:27 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:27.409298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752910b-1dba-4a83-bcb3-95750d407d7c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.415653 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 802/1587] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.416716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97d073-2349-4e22-a459-e550e211afa6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.419358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752910b-1dba-4a83-bcb3-95750d407d7c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.424880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54ea7b-d4bd-472a-bab4-8d76fb39052e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.428840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54ea7b-d4bd-472a-bab4-8d76fb39052e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.428840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54ea7b-d4bd-472a-bab4-8d76fb39052e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.428840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54ea7b-d4bd-472a-bab4-8d76fb39052e admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.428840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54ea7b-d4bd-472a-bab4-8d76fb39052e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.429264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97d073-2349-4e22-a459-e550e211afa6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.429360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54ea7b-d4bd-472a-bab4-8d76fb39052e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.433298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23374819-e524-4610-a8a0-47ca3bca8db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.433885 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23374819-e524-4610-a8a0-47ca3bca8db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.434167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23374819-e524-4610-a8a0-47ca3bca8db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.434439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23374819-e524-4610-a8a0-47ca3bca8db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.435076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23374819-e524-4610-a8a0-47ca3bca8db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.436764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23374819-e524-4610-a8a0-47ca3bca8db3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.447684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 803/1588] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.448485 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23374819-e524-4610-a8a0-47ca3bca8db3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.458601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23374819-e524-4610-a8a0-47ca3bca8db3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.463483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6e7e1-82c9-4a24-93fc-75168496d830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.464104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6e7e1-82c9-4a24-93fc-75168496d830 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.464383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6e7e1-82c9-4a24-93fc-75168496d830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.464656 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6e7e1-82c9-4a24-93fc-75168496d830 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.465314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6e7e1-82c9-4a24-93fc-75168496d830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.466885 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6e7e1-82c9-4a24-93fc-75168496d830 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.477632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 804/1589] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.478809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a958c-14d8-4941-98da-7ed5941ae368 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.479090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a958c-14d8-4941-98da-7ed5941ae368 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.479360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a958c-14d8-4941-98da-7ed5941ae368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.499650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ae0a958c-14d8-4941-98da-7ed5941ae368 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:27.500780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ae0a958c-14d8-4941-98da-7ed5941ae368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:27.508616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 805/1590] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:27.509438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a958c-14d8-4941-98da-7ed5941ae368 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.518639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a958c-14d8-4941-98da-7ed5941ae368 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.523302 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632feaa-9fb1-4dd3-9785-83ab9b3f8878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.523887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632feaa-9fb1-4dd3-9785-83ab9b3f8878 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.524160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632feaa-9fb1-4dd3-9785-83ab9b3f8878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.524433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632feaa-9fb1-4dd3-9785-83ab9b3f8878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.525113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632feaa-9fb1-4dd3-9785-83ab9b3f8878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.528059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 786/1591] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:27 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:27.539892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632feaa-9fb1-4dd3-9785-83ab9b3f8878 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.540020 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54ea7b-d4bd-472a-bab4-8d76fb39052e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.540148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 806/1592] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.540148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632feaa-9fb1-4dd3-9785-83ab9b3f8878 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.549218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632feaa-9fb1-4dd3-9785-83ab9b3f8878 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.594717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54ea7b-d4bd-472a-bab4-8d76fb39052e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.600208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd61ab-ebe3-4aed-b951-59d503c90397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.600846 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd61ab-ebe3-4aed-b951-59d503c90397 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.601158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd61ab-ebe3-4aed-b951-59d503c90397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.601437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fd61ab-ebe3-4aed-b951-59d503c90397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.602081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd61ab-ebe3-4aed-b951-59d503c90397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.603703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fd61ab-ebe3-4aed-b951-59d503c90397 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.606637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942af546-059a-4ad2-b65e-6db979cd61ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.607315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942af546-059a-4ad2-b65e-6db979cd61ae admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.607695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942af546-059a-4ad2-b65e-6db979cd61ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.608041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942af546-059a-4ad2-b65e-6db979cd61ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.608752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942af546-059a-4ad2-b65e-6db979cd61ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.611058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942af546-059a-4ad2-b65e-6db979cd61ae admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.614522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 787/1593] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.615174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd61ab-ebe3-4aed-b951-59d503c90397 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.621360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 807/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.622036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942af546-059a-4ad2-b65e-6db979cd61ae admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.627445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fd61ab-ebe3-4aed-b951-59d503c90397 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.631448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942af546-059a-4ad2-b65e-6db979cd61ae admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.632277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a07a45-6197-47a9-84b1-0952a69a94ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.632879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a07a45-6197-47a9-84b1-0952a69a94ef admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.633213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a07a45-6197-47a9-84b1-0952a69a94ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.633487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a07a45-6197-47a9-84b1-0952a69a94ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.634126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a07a45-6197-47a9-84b1-0952a69a94ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.635707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a07a45-6197-47a9-84b1-0952a69a94ef admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.636318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77162a3b-ae50-4b32-b80d-26aea5528aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.636979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162a3b-ae50-4b32-b80d-26aea5528aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.637343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162a3b-ae50-4b32-b80d-26aea5528aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.637673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162a3b-ae50-4b32-b80d-26aea5528aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.638306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77162a3b-ae50-4b32-b80d-26aea5528aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.639862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77162a3b-ae50-4b32-b80d-26aea5528aa8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.645881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 788/1595] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.652807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 808/1596] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.652979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a07a45-6197-47a9-84b1-0952a69a94ef admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.653112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77162a3b-ae50-4b32-b80d-26aea5528aa8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.654603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-77162a3b-ae50-4b32-b80d-26aea5528aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:27.660056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a07a45-6197-47a9-84b1-0952a69a94ef admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.664779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43adb0-ecfe-4ba0-8ed5-dd0d81eca900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.665522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43adb0-ecfe-4ba0-8ed5-dd0d81eca900 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.665804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43adb0-ecfe-4ba0-8ed5-dd0d81eca900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.666082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43adb0-ecfe-4ba0-8ed5-dd0d81eca900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.666725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43adb0-ecfe-4ba0-8ed5-dd0d81eca900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.668329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43adb0-ecfe-4ba0-8ed5-dd0d81eca900 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.668658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:27.683008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 789/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.683008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43adb0-ecfe-4ba0-8ed5-dd0d81eca900 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.690967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43adb0-ecfe-4ba0-8ed5-dd0d81eca900 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.695696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4deb7a-9997-4e46-b3b9-1005194e692d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.696495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4deb7a-9997-4e46-b3b9-1005194e692d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.696777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4deb7a-9997-4e46-b3b9-1005194e692d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.697082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4deb7a-9997-4e46-b3b9-1005194e692d admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.697710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4deb7a-9997-4e46-b3b9-1005194e692d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.699625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4deb7a-9997-4e46-b3b9-1005194e692d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.714866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77162a3b-ae50-4b32-b80d-26aea5528aa8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.726938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9820902-54a2-4763-a6f6-3a0773bbd353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.726938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9820902-54a2-4763-a6f6-3a0773bbd353 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.726938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9820902-54a2-4763-a6f6-3a0773bbd353 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.726938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9820902-54a2-4763-a6f6-3a0773bbd353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.729119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9820902-54a2-4763-a6f6-3a0773bbd353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.730532 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9820902-54a2-4763-a6f6-3a0773bbd353 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.740438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 809/1598] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:27.741239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9820902-54a2-4763-a6f6-3a0773bbd353 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.750105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9820902-54a2-4763-a6f6-3a0773bbd353 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.754830 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5639b-c267-43e7-a313-7a5f8f9d1bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.755408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5639b-c267-43e7-a313-7a5f8f9d1bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.755688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5639b-c267-43e7-a313-7a5f8f9d1bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.755955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5639b-c267-43e7-a313-7a5f8f9d1bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.756576 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5639b-c267-43e7-a313-7a5f8f9d1bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.758173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5639b-c267-43e7-a313-7a5f8f9d1bc8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.768634 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 810/1599] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.772704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5639b-c267-43e7-a313-7a5f8f9d1bc8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.778921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5639b-c267-43e7-a313-7a5f8f9d1bc8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.784252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d3532-6ced-4b43-97b9-f18c95b918c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.784838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d3532-6ced-4b43-97b9-f18c95b918c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.785153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d3532-6ced-4b43-97b9-f18c95b918c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.785425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d3532-6ced-4b43-97b9-f18c95b918c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.786075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d3532-6ced-4b43-97b9-f18c95b918c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.787651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d3532-6ced-4b43-97b9-f18c95b918c0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.797191 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 811/1600] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.798305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d3532-6ced-4b43-97b9-f18c95b918c0 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.807531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d3532-6ced-4b43-97b9-f18c95b918c0 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.812095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20d7fd-66b2-4455-bc1d-94c854ef673e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.812645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20d7fd-66b2-4455-bc1d-94c854ef673e admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.812942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20d7fd-66b2-4455-bc1d-94c854ef673e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.813224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20d7fd-66b2-4455-bc1d-94c854ef673e admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.813862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20d7fd-66b2-4455-bc1d-94c854ef673e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.815371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20d7fd-66b2-4455-bc1d-94c854ef673e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.825843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 812/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:27.826779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20d7fd-66b2-4455-bc1d-94c854ef673e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.835524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 790/1602] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:27 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:27.836532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4deb7a-9997-4e46-b3b9-1005194e692d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.836947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20d7fd-66b2-4455-bc1d-94c854ef673e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.842009 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9738a7-bd46-48c8-95eb-9f7b1b7fdfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.842620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9738a7-bd46-48c8-95eb-9f7b1b7fdfa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.842896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9738a7-bd46-48c8-95eb-9f7b1b7fdfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.843164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9738a7-bd46-48c8-95eb-9f7b1b7fdfa7 admin admin] PATH_INFO: `/v3/projects/c22d997be9cb4bbb8235eda18fa92074/users/28b31650305943b498f02c882baa0144/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.847359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4deb7a-9997-4e46-b3b9-1005194e692d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:27.853765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8c706-fa83-4e82-a509-1584132b6cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:27.854660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8c706-fa83-4e82-a509-1584132b6cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:27.854945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8c706-fa83-4e82-a509-1584132b6cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:27.855226 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8c706-fa83-4e82-a509-1584132b6cb4 admin admin] PATH_INFO: `/v3/projects/9f6ae51a6843404299c66e97cec51fec/users/bf9353bc130d45a99af9386ff9dd9a2f/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:27.858039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:27.859091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9738a7-bd46-48c8-95eb-9f7b1b7fdfa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22d997be9cb4bbb8235eda18fa92074, user_id=28b31650305943b498f02c882baa0144, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.861879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9738a7-bd46-48c8-95eb-9f7b1b7fdfa7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.871914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8c706-fa83-4e82-a509-1584132b6cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f6ae51a6843404299c66e97cec51fec, user_id=bf9353bc130d45a99af9386ff9dd9a2f, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:27.874651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8c706-fa83-4e82-a509-1584132b6cb4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:27.963190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 791/1603] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:27 2022] PUT /identity/v3/projects/9f6ae51a6843404299c66e97cec51fec/users/bf9353bc130d45a99af9386ff9dd9a2f/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:27.963374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 813/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:27 2022] PUT /identity/v3/projects/c22d997be9cb4bbb8235eda18fa92074/users/28b31650305943b498f02c882baa0144/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:27.964025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9738a7-bd46-48c8-95eb-9f7b1b7fdfa7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:27.972350 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8c706-fa83-4e82-a509-1584132b6cb4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:28.028491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9738a7-bd46-48c8-95eb-9f7b1b7fdfa7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.030047 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8c706-fa83-4e82-a509-1584132b6cb4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.034322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da37da-9ba6-4280-81cb-b31dcd914505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.034997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da37da-9ba6-4280-81cb-b31dcd914505 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.035281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da37da-9ba6-4280-81cb-b31dcd914505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.035605 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da37da-9ba6-4280-81cb-b31dcd914505 admin admin] PATH_INFO: `/v3/projects/cc58bc457b5b4d9bbb6df591429a8e47/users/61daf7f8bab64427a2edd6f98e720080/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.035716 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe29a21-7074-4999-92bd-536a3a8aa3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.036380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe29a21-7074-4999-92bd-536a3a8aa3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.036634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe29a21-7074-4999-92bd-536a3a8aa3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.036904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe29a21-7074-4999-92bd-536a3a8aa3ed admin admin] PATH_INFO: `/v3/projects/f3092f4cfd4e4372aee85c2567de2c5b/users/3353e6a3822a4b5d8e3fddcf1c5a3beb/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.052237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da37da-9ba6-4280-81cb-b31dcd914505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc58bc457b5b4d9bbb6df591429a8e47, user_id=61daf7f8bab64427a2edd6f98e720080, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.054722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe29a21-7074-4999-92bd-536a3a8aa3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3092f4cfd4e4372aee85c2567de2c5b, user_id=3353e6a3822a4b5d8e3fddcf1c5a3beb, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.056757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da37da-9ba6-4280-81cb-b31dcd914505 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:28.057639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe29a21-7074-4999-92bd-536a3a8aa3ed admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:28.139430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 792/1605] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:27 2022] PUT /identity/v3/projects/f3092f4cfd4e4372aee85c2567de2c5b/users/3353e6a3822a4b5d8e3fddcf1c5a3beb/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:28.140518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe29a21-7074-4999-92bd-536a3a8aa3ed admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:28.169557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 814/1606] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:27 2022] PUT /identity/v3/projects/cc58bc457b5b4d9bbb6df591429a8e47/users/61daf7f8bab64427a2edd6f98e720080/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:28.170396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da37da-9ba6-4280-81cb-b31dcd914505 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:28.209488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe29a21-7074-4999-92bd-536a3a8aa3ed admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.216452 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:28.236978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da37da-9ba6-4280-81cb-b31dcd914505 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.242065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdea49-de68-47e8-a8ff-41fc1005bb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.242730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdea49-de68-47e8-a8ff-41fc1005bb9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.243010 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdea49-de68-47e8-a8ff-41fc1005bb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.243353 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdea49-de68-47e8-a8ff-41fc1005bb9c admin admin] PATH_INFO: `/v3/projects/90177d2267614a389ae39922cba114e2/users/d6c2fc568ee74d35bba01c6ce294edeb/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.260159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdea49-de68-47e8-a8ff-41fc1005bb9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=90177d2267614a389ae39922cba114e2, user_id=d6c2fc568ee74d35bba01c6ce294edeb, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.263131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdea49-de68-47e8-a8ff-41fc1005bb9c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:28.269791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec1e4e-e66b-4685-b2ca-be081f916a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.270464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec1e4e-e66b-4685-b2ca-be081f916a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.270745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec1e4e-e66b-4685-b2ca-be081f916a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.271020 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec1e4e-e66b-4685-b2ca-be081f916a59 admin admin] PATH_INFO: `/v3/projects/33671f24dbc4478682f5e7b090bf48fa/users/c562afc432404b25bbd49dee793a8ce8/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.287164 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec1e4e-e66b-4685-b2ca-be081f916a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33671f24dbc4478682f5e7b090bf48fa, user_id=c562afc432404b25bbd49dee793a8ce8, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.288433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec1e4e-e66b-4685-b2ca-be081f916a59 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:28.291488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:28.332779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 815/1607] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:28 2022] PUT /identity/v3/projects/90177d2267614a389ae39922cba114e2/users/d6c2fc568ee74d35bba01c6ce294edeb/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:28.336550 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdea49-de68-47e8-a8ff-41fc1005bb9c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:28.389964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 793/1608] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:28 2022] PUT /identity/v3/projects/33671f24dbc4478682f5e7b090bf48fa/users/c562afc432404b25bbd49dee793a8ce8/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:28.391464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d5585-d10f-4938-932b-94404a50f9ed None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.391797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d5585-d10f-4938-932b-94404a50f9ed None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.392581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d5585-d10f-4938-932b-94404a50f9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.404136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdea49-de68-47e8-a8ff-41fc1005bb9c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.414068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-234d5585-d10f-4938-932b-94404a50f9ed None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:28.415382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-234d5585-d10f-4938-932b-94404a50f9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:28.472178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a94ab3-d76f-4f9f-96ef-a08a5d11fa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.472791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a94ab3-d76f-4f9f-96ef-a08a5d11fa21 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.473108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a94ab3-d76f-4f9f-96ef-a08a5d11fa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.473445 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a94ab3-d76f-4f9f-96ef-a08a5d11fa21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.474081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a94ab3-d76f-4f9f-96ef-a08a5d11fa21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.475730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a94ab3-d76f-4f9f-96ef-a08a5d11fa21 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:28.482272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 794/1609] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:28.483559 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a5001-0d93-4744-8ac9-13aac5243d43 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.483850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a5001-0d93-4744-8ac9-13aac5243d43 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.484126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a5001-0d93-4744-8ac9-13aac5243d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.489316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 816/1610] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:28.490798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97381bf8-6d17-4658-9b2f-1ed142b8b160 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.491325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97381bf8-6d17-4658-9b2f-1ed142b8b160 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.491744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97381bf8-6d17-4658-9b2f-1ed142b8b160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.506894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7c0a5001-0d93-4744-8ac9-13aac5243d43 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:28.508124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7c0a5001-0d93-4744-8ac9-13aac5243d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:28.512967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-97381bf8-6d17-4658-9b2f-1ed142b8b160 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:28.514177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-97381bf8-6d17-4658-9b2f-1ed142b8b160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:28.515998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 795/1611] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:28.517462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963f43c-3952-4382-b197-d27b5f058955 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.521090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963f43c-3952-4382-b197-d27b5f058955 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.521090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963f43c-3952-4382-b197-d27b5f058955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.525060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 817/1612] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:28.526163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7dcdf-00c6-4ab6-b3ba-1745ec7e1fc2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.526261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7dcdf-00c6-4ab6-b3ba-1745ec7e1fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.526515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7dcdf-00c6-4ab6-b3ba-1745ec7e1fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.547208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-37c7dcdf-00c6-4ab6-b3ba-1745ec7e1fc2 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:28.549007 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-37c7dcdf-00c6-4ab6-b3ba-1745ec7e1fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:28.554948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3963f43c-3952-4382-b197-d27b5f058955 None None] MFA Rules not processed for user `61daf7f8bab64427a2edd6f98e720080`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:28.556216 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3963f43c-3952-4382-b197-d27b5f058955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:28.561146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 818/1613] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:28.561146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ef6e8-d9cb-4f79-94a7-a1ec6cc9b2e9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.561146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ef6e8-d9cb-4f79-94a7-a1ec6cc9b2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.561146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ef6e8-d9cb-4f79-94a7-a1ec6cc9b2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.592709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-440ef6e8-d9cb-4f79-94a7-a1ec6cc9b2e9 None None] MFA Rules not processed for user `c562afc432404b25bbd49dee793a8ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:28.594091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-440ef6e8-d9cb-4f79-94a7-a1ec6cc9b2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:28.641223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 796/1614] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:28 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:28.642720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963f43c-3952-4382-b197-d27b5f058955 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:28.648259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:28.656349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963f43c-3952-4382-b197-d27b5f058955 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.662113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8995988-28f4-459d-9d4e-0dd1f7341258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.662845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8995988-28f4-459d-9d4e-0dd1f7341258 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.663130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8995988-28f4-459d-9d4e-0dd1f7341258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.664293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8995988-28f4-459d-9d4e-0dd1f7341258 admin admin] PATH_INFO: `/v3/projects/91fc5be35c9f4a7283162c7565e04259/users/8e1b09cd8e4342079a0b173035ee505b/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.673852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 819/1615] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:28.675185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ef6e8-d9cb-4f79-94a7-a1ec6cc9b2e9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:28.684343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8995988-28f4-459d-9d4e-0dd1f7341258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91fc5be35c9f4a7283162c7565e04259, user_id=8e1b09cd8e4342079a0b173035ee505b, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.687561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8995988-28f4-459d-9d4e-0dd1f7341258 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:28.736409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ef6e8-d9cb-4f79-94a7-a1ec6cc9b2e9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.740164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be71d29-2366-46e6-811e-2a42a8a8f972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.740797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be71d29-2366-46e6-811e-2a42a8a8f972 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.741547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be71d29-2366-46e6-811e-2a42a8a8f972 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.741547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be71d29-2366-46e6-811e-2a42a8a8f972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.750229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be71d29-2366-46e6-811e-2a42a8a8f972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.751933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be71d29-2366-46e6-811e-2a42a8a8f972 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:28.761276 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 820/1616] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:28.762091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be71d29-2366-46e6-811e-2a42a8a8f972 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:28.770650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be71d29-2366-46e6-811e-2a42a8a8f972 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.773002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 797/1617] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:28 2022] PUT /identity/v3/projects/91fc5be35c9f4a7283162c7565e04259/users/8e1b09cd8e4342079a0b173035ee505b/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:28.775841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8995988-28f4-459d-9d4e-0dd1f7341258 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:28.824733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873aec1-09c7-4f17-94d4-80fc8f95c32b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.825367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873aec1-09c7-4f17-94d4-80fc8f95c32b service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.825649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873aec1-09c7-4f17-94d4-80fc8f95c32b service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.825922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873aec1-09c7-4f17-94d4-80fc8f95c32b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.839821 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8995988-28f4-459d-9d4e-0dd1f7341258 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.843522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92041afe-b6c9-4f4e-9713-064960307bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.844135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92041afe-b6c9-4f4e-9713-064960307bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.844929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92041afe-b6c9-4f4e-9713-064960307bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.844929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92041afe-b6c9-4f4e-9713-064960307bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.854784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:28.903458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873aec1-09c7-4f17-94d4-80fc8f95c32b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.904934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2873aec1-09c7-4f17-94d4-80fc8f95c32b service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:28.914603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92041afe-b6c9-4f4e-9713-064960307bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.916492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92041afe-b6c9-4f4e-9713-064960307bd1 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:28.918850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 821/1618] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:28.919933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873aec1-09c7-4f17-94d4-80fc8f95c32b service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:28.927373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 798/1619] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:28.928371 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92041afe-b6c9-4f4e-9713-064960307bd1 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:28.930112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873aec1-09c7-4f17-94d4-80fc8f95c32b service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.934995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6a12-c1f7-45a8-a448-406965583860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.935890 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6a12-c1f7-45a8-a448-406965583860 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.936322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6a12-c1f7-45a8-a448-406965583860 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.937052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6a12-c1f7-45a8-a448-406965583860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.938314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92041afe-b6c9-4f4e-9713-064960307bd1 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.944294 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf5a8f-4763-4531-9af2-7d0e83ea3a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.944999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf5a8f-4763-4531-9af2-7d0e83ea3a8d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.945404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf5a8f-4763-4531-9af2-7d0e83ea3a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.945681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf5a8f-4763-4531-9af2-7d0e83ea3a8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.946444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf5a8f-4763-4531-9af2-7d0e83ea3a8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.948497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf5a8f-4763-4531-9af2-7d0e83ea3a8d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:28.950904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb6a12-c1f7-45a8-a448-406965583860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.952785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb6a12-c1f7-45a8-a448-406965583860 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:28.966607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 822/1620] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:28.967976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7e5ad-d9af-484f-8fdb-df56567c9e53 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.968267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7e5ad-d9af-484f-8fdb-df56567c9e53 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.968543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7e5ad-d9af-484f-8fdb-df56567c9e53 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.970581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 823/1621] 10.222.0.17 () {54 vars in 962 bytes} [Thu Aug 11 09:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:59:28.973036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7e5ad-d9af-484f-8fdb-df56567c9e53 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:28.987761 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7e5ad-d9af-484f-8fdb-df56567c9e53 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:28.994730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d770e-716f-4da8-a01f-5b31f76846dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:28.995347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d770e-716f-4da8-a01f-5b31f76846dd admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:28.995634 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d770e-716f-4da8-a01f-5b31f76846dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:28.995910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d770e-716f-4da8-a01f-5b31f76846dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:28.996565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d770e-716f-4da8-a01f-5b31f76846dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:28.998197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d770e-716f-4da8-a01f-5b31f76846dd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:29.012980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 824/1622] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:29.012980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586731a0-b407-4208-8816-2718d50bf00e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:29.012980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586731a0-b407-4208-8816-2718d50bf00e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:29.012980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586731a0-b407-4208-8816-2718d50bf00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:29.034173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:29.053934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-586731a0-b407-4208-8816-2718d50bf00e None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:29.055143 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-586731a0-b407-4208-8816-2718d50bf00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:29.069427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 799/1623] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:28 2022] POST /identity/v3/projects => generated 405 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:29.069427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf5a8f-4763-4531-9af2-7d0e83ea3a8d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:29.162399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf5a8f-4763-4531-9af2-7d0e83ea3a8d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:29.177768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6588a-9b13-469c-a493-2c3119a18f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:29.178346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6588a-9b13-469c-a493-2c3119a18f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:29.178624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6588a-9b13-469c-a493-2c3119a18f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:29.178900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6588a-9b13-469c-a493-2c3119a18f5b admin admin] PATH_INFO: `/v3/projects/91fc5be35c9f4a7283162c7565e04259/users/8e1b09cd8e4342079a0b173035ee505b/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:29.187895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6588a-9b13-469c-a493-2c3119a18f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=91fc5be35c9f4a7283162c7565e04259, user_id=8e1b09cd8e4342079a0b173035ee505b, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:29.191543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 825/1624] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Aug 11 09:59:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:29.193865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586731a0-b407-4208-8816-2718d50bf00e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:29.196160 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6588a-9b13-469c-a493-2c3119a18f5b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:29.225524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586731a0-b407-4208-8816-2718d50bf00e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:29.229627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 800/1625] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:29 2022] PUT /identity/v3/projects/91fc5be35c9f4a7283162c7565e04259/users/8e1b09cd8e4342079a0b173035ee505b/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:29.231266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6588a-9b13-469c-a493-2c3119a18f5b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:29.350004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6588a-9b13-469c-a493-2c3119a18f5b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:29.353852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60627f6-f4d6-44b4-aae1-98f883d1b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:29.353852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60627f6-f4d6-44b4-aae1-98f883d1b2da admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:29.353852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60627f6-f4d6-44b4-aae1-98f883d1b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:29.353852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60627f6-f4d6-44b4-aae1-98f883d1b2da admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:29.354368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60627f6-f4d6-44b4-aae1-98f883d1b2da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:29.355917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60627f6-f4d6-44b4-aae1-98f883d1b2da admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:29.368722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faeaf82-c56d-4bc6-802c-2a41e953203a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:29.369390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faeaf82-c56d-4bc6-802c-2a41e953203a service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:29.369677 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faeaf82-c56d-4bc6-802c-2a41e953203a service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:29.369953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faeaf82-c56d-4bc6-802c-2a41e953203a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:29.391935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:29.468156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faeaf82-c56d-4bc6-802c-2a41e953203a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:29.471805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faeaf82-c56d-4bc6-802c-2a41e953203a service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:29.474398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 826/1626] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:29 2022] POST /identity/v3/users => generated 309 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:29.477750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e0238-6046-42ec-bdcf-8a5ea23fbf27 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:29.478035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e0238-6046-42ec-bdcf-8a5ea23fbf27 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:29.478308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e0238-6046-42ec-bdcf-8a5ea23fbf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:29.497258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 801/1627] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:29.502574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faeaf82-c56d-4bc6-802c-2a41e953203a service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:29.531753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faeaf82-c56d-4bc6-802c-2a41e953203a service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:29.540335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-917e0238-6046-42ec-bdcf-8a5ea23fbf27 None None] MFA Rules not processed for user `8e1b09cd8e4342079a0b173035ee505b`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:29.543006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-917e0238-6046-42ec-bdcf-8a5ea23fbf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:29.566141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4e3d3-defe-4f97-93b6-b994b35d3287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:29.569194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e3d3-defe-4f97-93b6-b994b35d3287 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:29.569487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e3d3-defe-4f97-93b6-b994b35d3287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:29.569773 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4e3d3-defe-4f97-93b6-b994b35d3287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:29.575154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4e3d3-defe-4f97-93b6-b994b35d3287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:29.585752 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4e3d3-defe-4f97-93b6-b994b35d3287 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:29.636679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 802/1628] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:29.639406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4e3d3-defe-4f97-93b6-b994b35d3287 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:29.673817 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4e3d3-defe-4f97-93b6-b994b35d3287 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:29.695705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a866ae-e339-4287-9a52-62d629b65417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:29.696389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a866ae-e339-4287-9a52-62d629b65417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:29.696673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a866ae-e339-4287-9a52-62d629b65417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:29.696950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a866ae-e339-4287-9a52-62d629b65417 admin admin] PATH_INFO: `/v3/projects/0c12edaaacd14c919d1a57b46494ed71/users/fc2747bf30c840bbb1ef47f38290529f/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:29.714006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 827/1629] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:29.728412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e0238-6046-42ec-bdcf-8a5ea23fbf27 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:29.734730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:29.741845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a866ae-e339-4287-9a52-62d629b65417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c12edaaacd14c919d1a57b46494ed71, user_id=fc2747bf30c840bbb1ef47f38290529f, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:29.745791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a866ae-e339-4287-9a52-62d629b65417 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:29.805727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e0238-6046-42ec-bdcf-8a5ea23fbf27 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:29.810085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8479698-bf76-4ccf-a184-fd253ef2d307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:29.810835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8479698-bf76-4ccf-a184-fd253ef2d307 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:29.811029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8479698-bf76-4ccf-a184-fd253ef2d307 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:29.811268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8479698-bf76-4ccf-a184-fd253ef2d307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:29.820715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8479698-bf76-4ccf-a184-fd253ef2d307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:29.822175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8479698-bf76-4ccf-a184-fd253ef2d307 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:29.832360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 828/1630] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 09:59:29.864975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 803/1631] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:29 2022] PUT /identity/v3/projects/0c12edaaacd14c919d1a57b46494ed71/users/fc2747bf30c840bbb1ef47f38290529f/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:29.865217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8479698-bf76-4ccf-a184-fd253ef2d307 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:29.874196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8479698-bf76-4ccf-a184-fd253ef2d307 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:29.900841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79f375-cb0e-43d7-add5-ec08d96601d5 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:29.901157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79f375-cb0e-43d7-add5-ec08d96601d5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:29.901440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79f375-cb0e-43d7-add5-ec08d96601d5 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:29.903431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 804/1632] 10.222.0.17 () {54 vars in 912 bytes} [Thu Aug 11 09:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 09:59:29.909772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f6202-2f5d-47c6-a84e-5e8245512a6d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:29.910058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f6202-2f5d-47c6-a84e-5e8245512a6d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:29.910338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f6202-2f5d-47c6-a84e-5e8245512a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:29.957292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7520b-edda-45cd-96b4-15143bb32336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:29.958142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7520b-edda-45cd-96b4-15143bb32336 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:29.958435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7520b-edda-45cd-96b4-15143bb32336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:29.958711 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7520b-edda-45cd-96b4-15143bb32336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:29.959426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7520b-edda-45cd-96b4-15143bb32336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:29.961787 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7520b-edda-45cd-96b4-15143bb32336 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:29.977488 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 829/1633] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:29.982747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7520b-edda-45cd-96b4-15143bb32336 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:29.995450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7520b-edda-45cd-96b4-15143bb32336 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:30.005284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf27d3-48b3-4ba8-8f91-ba673359d27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:30.006146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf27d3-48b3-4ba8-8f91-ba673359d27d admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:30.006627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf27d3-48b3-4ba8-8f91-ba673359d27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:30.006982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf27d3-48b3-4ba8-8f91-ba673359d27d admin admin] PATH_INFO: `/v3/projects/0c12edaaacd14c919d1a57b46494ed71/users/fc2747bf30c840bbb1ef47f38290529f/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:30.008436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8b8f6202-2f5d-47c6-a84e-5e8245512a6d None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:30.009506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf27d3-48b3-4ba8-8f91-ba673359d27d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c12edaaacd14c919d1a57b46494ed71, user_id=fc2747bf30c840bbb1ef47f38290529f, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:30.013973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8b8f6202-2f5d-47c6-a84e-5e8245512a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:30.014114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf27d3-48b3-4ba8-8f91-ba673359d27d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:30.043638 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 830/1634] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:29 2022] PUT /identity/v3/projects/0c12edaaacd14c919d1a57b46494ed71/users/fc2747bf30c840bbb1ef47f38290529f/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:30.045955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdee43-b795-4521-8906-377305bcdfa2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:30.045955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdee43-b795-4521-8906-377305bcdfa2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:30.046206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdee43-b795-4521-8906-377305bcdfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:30.076392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a3fdee43-b795-4521-8906-377305bcdfa2 None None] MFA Rules not processed for user `fc2747bf30c840bbb1ef47f38290529f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:30.077775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a3fdee43-b795-4521-8906-377305bcdfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:30.106308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 805/1635] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 09:59:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:59:30.120512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f6202-2f5d-47c6-a84e-5e8245512a6d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:30.245684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f6202-2f5d-47c6-a84e-5e8245512a6d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:30.246561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 831/1636] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:30 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:30.247443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdee43-b795-4521-8906-377305bcdfa2 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:30.306905 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:30.355263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bd8f6-1b98-420c-a962-ed92bdd05ec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:30.356049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bd8f6-1b98-420c-a962-ed92bdd05ec5 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:30.356439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bd8f6-1b98-420c-a962-ed92bdd05ec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:30.356981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bd8f6-1b98-420c-a962-ed92bdd05ec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:30.379877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdee43-b795-4521-8906-377305bcdfa2 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:30.380593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4881a-f2c7-4e35-a617-6d78daa165ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:30.381257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4881a-f2c7-4e35-a617-6d78daa165ef service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:30.381615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4881a-f2c7-4e35-a617-6d78daa165ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:30.381969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4881a-f2c7-4e35-a617-6d78daa165ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:30.480597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:30.481922 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bd8f6-1b98-420c-a962-ed92bdd05ec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:30.486263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bd8f6-1b98-420c-a962-ed92bdd05ec5 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:30.506449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 806/1637] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 09:59:30.516189 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4881a-f2c7-4e35-a617-6d78daa165ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:30.517847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4881a-f2c7-4e35-a617-6d78daa165ef service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:30.530012 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 832/1638] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:31.241703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bd8f6-1b98-420c-a962-ed92bdd05ec5 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:31.258457 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bd8f6-1b98-420c-a962-ed92bdd05ec5 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:31.263898 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a2d8c-c16b-46a6-be54-635fa0122491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:31.264595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2d8c-c16b-46a6-be54-635fa0122491 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:31.265114 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2d8c-c16b-46a6-be54-635fa0122491 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:31.265510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2d8c-c16b-46a6-be54-635fa0122491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:31.281513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a2d8c-c16b-46a6-be54-635fa0122491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:31.284769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a2d8c-c16b-46a6-be54-635fa0122491 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:31.299919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 807/1639] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:31 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:31.642454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4881a-f2c7-4e35-a617-6d78daa165ef service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:31.756072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4881a-f2c7-4e35-a617-6d78daa165ef service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:31.782899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:31.782899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:31.782899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:31.782899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] PATH_INFO: `/v3/users/61daf7f8bab64427a2edd6f98e720080` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:31.784271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61daf7f8bab64427a2edd6f98e720080)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:31.784271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: WARNING py.warnings [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:59:31.784271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: warnings.warn(msg) Aug 11 09:59:31.784271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: Aug 11 09:59:31.784271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:32.245985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61daf7f8bab64427a2edd6f98e720080'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:32.245985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61daf7f8bab64427a2edd6f98e720080'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:32.260413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad48730-770f-4295-af51-f0e24a634a95 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:32.260413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad48730-770f-4295-af51-f0e24a634a95 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:32.260413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad48730-770f-4295-af51-f0e24a634a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:32.264297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61daf7f8bab64427a2edd6f98e720080'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:32.264297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61daf7f8bab64427a2edd6f98e720080'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:32.264297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:32.264297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2ab97456-1c61-4809-9600-09a5e359d82d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61daf7f8bab64427a2edd6f98e720080'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:32.286924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 833/1640] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 09:59:31 2022] DELETE /identity/v3/users/61daf7f8bab64427a2edd6f98e720080 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:32.288359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3ad48730-770f-4295-af51-f0e24a634a95 None None] MFA Rules not processed for user `3353e6a3822a4b5d8e3fddcf1c5a3beb`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:32.289655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3ad48730-770f-4295-af51-f0e24a634a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:32.298512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00839-4633-4ea3-89d7-1ee085e6be47 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:32.298805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00839-4633-4ea3-89d7-1ee085e6be47 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:32.299082 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b00839-4633-4ea3-89d7-1ee085e6be47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:32.340080 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c7b00839-4633-4ea3-89d7-1ee085e6be47 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:32.341375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c7b00839-4633-4ea3-89d7-1ee085e6be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:32.477592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 834/1641] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:32.511774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 808/1642] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:32 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:59:32.513269 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b00839-4633-4ea3-89d7-1ee085e6be47 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:32.514828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c7b00839-4633-4ea3-89d7-1ee085e6be47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:32.535206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b00839-4633-4ea3-89d7-1ee085e6be47 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:32.554143 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73086362-5ee2-4c9e-83d7-743a5d20871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:32.554980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73086362-5ee2-4c9e-83d7-743a5d20871d admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:32.555272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73086362-5ee2-4c9e-83d7-743a5d20871d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:32.555545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73086362-5ee2-4c9e-83d7-743a5d20871d admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:32.562405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73086362-5ee2-4c9e-83d7-743a5d20871d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:32.563525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73086362-5ee2-4c9e-83d7-743a5d20871d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:32.573072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad48730-770f-4295-af51-f0e24a634a95 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:32.574539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3ad48730-770f-4295-af51-f0e24a634a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:32.701374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:32.708483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 835/1643] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:32 2022] POST /identity/v3/projects => generated 403 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:32.712480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73086362-5ee2-4c9e-83d7-743a5d20871d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:32.726337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad48730-770f-4295-af51-f0e24a634a95 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:32.828584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73086362-5ee2-4c9e-83d7-743a5d20871d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:32.834895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7ade9-e301-42ec-b21f-9c87db75edfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:32.843598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7ade9-e301-42ec-b21f-9c87db75edfc admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:32.843598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7ade9-e301-42ec-b21f-9c87db75edfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:32.843598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7ade9-e301-42ec-b21f-9c87db75edfc admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:32.843598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7ade9-e301-42ec-b21f-9c87db75edfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:32.843598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7ade9-e301-42ec-b21f-9c87db75edfc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:32.850801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f36b90-a5ce-4404-bff8-c7216ed46907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:32.851426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f36b90-a5ce-4404-bff8-c7216ed46907 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:32.851717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f36b90-a5ce-4404-bff8-c7216ed46907 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:32.851999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f36b90-a5ce-4404-bff8-c7216ed46907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:32.867407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f36b90-a5ce-4404-bff8-c7216ed46907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:32.868839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f36b90-a5ce-4404-bff8-c7216ed46907 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:32.882102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 809/1644] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 09:59:32.883435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bebbd-cdbd-46fd-affc-425d0c44541e None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:32.883723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bebbd-cdbd-46fd-affc-425d0c44541e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:32.884165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bebbd-cdbd-46fd-affc-425d0c44541e None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:32.885643 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 810/1645] 10.222.0.17 () {54 vars in 962 bytes} [Thu Aug 11 09:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:59:32.895487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0a2c7-4d3f-483b-a0a8-5804fd51c776 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:32.896863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0a2c7-4d3f-483b-a0a8-5804fd51c776 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:32.897382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0a2c7-4d3f-483b-a0a8-5804fd51c776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:32.931868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1fd0a2c7-4d3f-483b-a0a8-5804fd51c776 None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:32.933857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1fd0a2c7-4d3f-483b-a0a8-5804fd51c776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:33.023549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 836/1646] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:32 2022] POST /identity/v3/users => generated 308 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:59:33.040782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7ade9-e301-42ec-b21f-9c87db75edfc admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:33.084466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7ade9-e301-42ec-b21f-9c87db75edfc admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:33.103485 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b53c4-c0d5-46a7-a03e-094d6d6ad8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:33.104363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b53c4-c0d5-46a7-a03e-094d6d6ad8bf admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:33.106702 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b53c4-c0d5-46a7-a03e-094d6d6ad8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:33.107109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b53c4-c0d5-46a7-a03e-094d6d6ad8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:33.108021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b53c4-c0d5-46a7-a03e-094d6d6ad8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:33.110399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b53c4-c0d5-46a7-a03e-094d6d6ad8bf admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:33.132833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 837/1647] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:33.134994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b53c4-c0d5-46a7-a03e-094d6d6ad8bf admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:33.172482 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b53c4-c0d5-46a7-a03e-094d6d6ad8bf admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:33.183238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b9a6-247d-46b7-8617-792b04247251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:33.185000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b9a6-247d-46b7-8617-792b04247251 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:33.185580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b9a6-247d-46b7-8617-792b04247251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:33.186067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443b9a6-247d-46b7-8617-792b04247251 admin admin] PATH_INFO: `/v3/projects/b0295d2b57434ce197af9875347cfb76/users/089508b9ca2f42209f6ae189e4e3ad90/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:33.196618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 811/1648] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Aug 11 09:59:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:33.213757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f6e65-d635-4d6e-98c0-e83c1e065f22 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:33.214111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f6e65-d635-4d6e-98c0-e83c1e065f22 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:33.214321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f6e65-d635-4d6e-98c0-e83c1e065f22 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:33.216137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 812/1649] 10.222.0.17 () {54 vars in 912 bytes} [Thu Aug 11 09:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:59:33.226633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f52a0-6ca6-492d-b28e-a2f443e5ac55 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:33.226942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f52a0-6ca6-492d-b28e-a2f443e5ac55 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:33.227209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f52a0-6ca6-492d-b28e-a2f443e5ac55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:33.239007 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443b9a6-247d-46b7-8617-792b04247251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0295d2b57434ce197af9875347cfb76, user_id=089508b9ca2f42209f6ae189e4e3ad90, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:33.245412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443b9a6-247d-46b7-8617-792b04247251 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:33.287867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-038f52a0-6ca6-492d-b28e-a2f443e5ac55 None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:33.287867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-038f52a0-6ca6-492d-b28e-a2f443e5ac55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:33.305329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 813/1650] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 09:59:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:59:33.336585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f52a0-6ca6-492d-b28e-a2f443e5ac55 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:33.339280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-038f52a0-6ca6-492d-b28e-a2f443e5ac55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:33.360848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 838/1651] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:33 2022] PUT /identity/v3/projects/b0295d2b57434ce197af9875347cfb76/users/089508b9ca2f42209f6ae189e4e3ad90/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:33.374182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b9a6-247d-46b7-8617-792b04247251 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:33.524825 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443b9a6-247d-46b7-8617-792b04247251 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:33.545359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bfe22-dc61-4251-a01f-da56845eb6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:33.547238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f52a0-6ca6-492d-b28e-a2f443e5ac55 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:33.550265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bfe22-dc61-4251-a01f-da56845eb6cc admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:33.551952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bfe22-dc61-4251-a01f-da56845eb6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:33.555076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bfe22-dc61-4251-a01f-da56845eb6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:33.559741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bfe22-dc61-4251-a01f-da56845eb6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:33.568312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bfe22-dc61-4251-a01f-da56845eb6cc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:33.597598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 839/1652] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:33 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 09:59:33.601579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bfe22-dc61-4251-a01f-da56845eb6cc admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:33.621086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bfe22-dc61-4251-a01f-da56845eb6cc admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:33.646004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab499d9-baf3-4aef-b108-8f68e372424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:33.647098 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab499d9-baf3-4aef-b108-8f68e372424a admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:33.647573 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab499d9-baf3-4aef-b108-8f68e372424a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:33.648035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab499d9-baf3-4aef-b108-8f68e372424a admin admin] PATH_INFO: `/v3/projects/b0295d2b57434ce197af9875347cfb76/users/089508b9ca2f42209f6ae189e4e3ad90/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:33.657449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:33.658876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab499d9-baf3-4aef-b108-8f68e372424a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0295d2b57434ce197af9875347cfb76, user_id=089508b9ca2f42209f6ae189e4e3ad90, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:33.661690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab499d9-baf3-4aef-b108-8f68e372424a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:33.687584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 840/1653] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:33 2022] PUT /identity/v3/projects/b0295d2b57434ce197af9875347cfb76/users/089508b9ca2f42209f6ae189e4e3ad90/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:33.688896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cf5eb-7a14-46a1-9e1f-fd264ce1f8d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:33.690298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cf5eb-7a14-46a1-9e1f-fd264ce1f8d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:33.690592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cf5eb-7a14-46a1-9e1f-fd264ce1f8d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:33.690873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cf5eb-7a14-46a1-9e1f-fd264ce1f8d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:33.808004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:33.926728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cf5eb-7a14-46a1-9e1f-fd264ce1f8d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:33.929171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cf5eb-7a14-46a1-9e1f-fd264ce1f8d2 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:33.954981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 814/1654] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:34.567739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab499d9-baf3-4aef-b108-8f68e372424a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:34.570102 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-aab499d9-baf3-4aef-b108-8f68e372424a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:34.734885 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab499d9-baf3-4aef-b108-8f68e372424a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:34.743518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:34.745755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:34.746253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:34.746748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] PATH_INFO: `/v3/projects/cc58bc457b5b4d9bbb6df591429a8e47` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:34.748308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc58bc457b5b4d9bbb6df591429a8e47)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:34.754622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:34.980111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc58bc457b5b4d9bbb6df591429a8e47', 'request_id': 'req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:34.988298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc58bc457b5b4d9bbb6df591429a8e47', 'request_id': 'req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:34.998583 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc58bc457b5b4d9bbb6df591429a8e47', 'request_id': 'req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:35.134833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] The token cache is being invalidate because project cc58bc457b5b4d9bbb6df591429a8e47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 09:59:35.140070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:35.140070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:35.140070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 841/1655] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 09:59:34 2022] DELETE /identity/v3/projects/cc58bc457b5b4d9bbb6df591429a8e47 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 09:59:35.143279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cf5eb-7a14-46a1-9e1f-fd264ce1f8d2 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:35.160978 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cf5eb-7a14-46a1-9e1f-fd264ce1f8d2 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:35.305605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2e13f-4a40-454a-976d-9f7ec86ea144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:35.308405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2e13f-4a40-454a-976d-9f7ec86ea144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:35.308977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2e13f-4a40-454a-976d-9f7ec86ea144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:35.309355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2e13f-4a40-454a-976d-9f7ec86ea144 admin admin] PATH_INFO: `/v3/users/fc2747bf30c840bbb1ef47f38290529f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:35.315575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2e13f-4a40-454a-976d-9f7ec86ea144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc2747bf30c840bbb1ef47f38290529f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:35.319796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2e13f-4a40-454a-976d-9f7ec86ea144 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:35.590059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bbd2e13f-4a40-454a-976d-9f7ec86ea144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc2747bf30c840bbb1ef47f38290529f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:35.591339 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bbd2e13f-4a40-454a-976d-9f7ec86ea144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc2747bf30c840bbb1ef47f38290529f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:35.592163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bbd2e13f-4a40-454a-976d-9f7ec86ea144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc2747bf30c840bbb1ef47f38290529f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:35.756718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bbd2e13f-4a40-454a-976d-9f7ec86ea144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc2747bf30c840bbb1ef47f38290529f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:35.796190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bbd2e13f-4a40-454a-976d-9f7ec86ea144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc2747bf30c840bbb1ef47f38290529f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:35.842853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 815/1656] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 09:59:35 2022] DELETE /identity/v3/users/fc2747bf30c840bbb1ef47f38290529f => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:37.150186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:37.151752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:37.175734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46f12e-a2a7-49f3-b180-b46cbfce3f7d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:37.276136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:37.341466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:37.341466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:37.341466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:37.341466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] PATH_INFO: `/v3/projects/0c12edaaacd14c919d1a57b46494ed71` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:37.341776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c12edaaacd14c919d1a57b46494ed71)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:37.341776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:37.527618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c12edaaacd14c919d1a57b46494ed71', 'request_id': 'req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:37.535258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c12edaaacd14c919d1a57b46494ed71', 'request_id': 'req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:37.550817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c12edaaacd14c919d1a57b46494ed71', 'request_id': 'req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:37.671627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] The token cache is being invalidate because project 0c12edaaacd14c919d1a57b46494ed71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 09:59:37.671927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:37.674779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9d7ac093-c2df-48c3-8c92-0bf867fa4c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:37.679746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 842/1657] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 09:59:37 2022] DELETE /identity/v3/projects/0c12edaaacd14c919d1a57b46494ed71 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:37.681594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cc50f-2a2f-4cf4-9974-c29c3a1717d3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:37.682088 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cc50f-2a2f-4cf4-9974-c29c3a1717d3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:37.682199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cc50f-2a2f-4cf4-9974-c29c3a1717d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:37.702503 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:37.726689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-320cc50f-2a2f-4cf4-9974-c29c3a1717d3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:37.727953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-320cc50f-2a2f-4cf4-9974-c29c3a1717d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:37.851616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 816/1658] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:37.859435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3edeb-29da-47b5-b84f-9a81f53aa147 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:37.859740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3edeb-29da-47b5-b84f-9a81f53aa147 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:37.860194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b3edeb-29da-47b5-b84f-9a81f53aa147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:37.896838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d4b3edeb-29da-47b5-b84f-9a81f53aa147 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:37.898482 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d4b3edeb-29da-47b5-b84f-9a81f53aa147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:37.910976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 843/1659] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:37.915308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cc50f-2a2f-4cf4-9974-c29c3a1717d3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:37.930005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cc50f-2a2f-4cf4-9974-c29c3a1717d3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:37.949599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61044df7-e482-4b1d-b887-48ce0b737a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:37.950649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61044df7-e482-4b1d-b887-48ce0b737a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:37.951096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61044df7-e482-4b1d-b887-48ce0b737a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:37.951511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61044df7-e482-4b1d-b887-48ce0b737a55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:37.952324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61044df7-e482-4b1d-b887-48ce0b737a55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:37.953126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61044df7-e482-4b1d-b887-48ce0b737a55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:37.956730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61044df7-e482-4b1d-b887-48ce0b737a55 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:37.992580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 817/1660] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:37.993429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3edeb-29da-47b5-b84f-9a81f53aa147 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:38.007927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b3edeb-29da-47b5-b84f-9a81f53aa147 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:38.021374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4339c59-0f06-41a6-99bd-371e46ca8870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:38.022125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339c59-0f06-41a6-99bd-371e46ca8870 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:38.025126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339c59-0f06-41a6-99bd-371e46ca8870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:38.025620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339c59-0f06-41a6-99bd-371e46ca8870 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:38.026416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4339c59-0f06-41a6-99bd-371e46ca8870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:38.031220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4339c59-0f06-41a6-99bd-371e46ca8870 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:38.053347 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cfe8d-de22-46c9-9108-a038dd881279 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:38.053638 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cfe8d-de22-46c9-9108-a038dd881279 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:38.053920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cfe8d-de22-46c9-9108-a038dd881279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:38.115386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-833cfe8d-de22-46c9-9108-a038dd881279 None None] MFA Rules not processed for user `089508b9ca2f42209f6ae189e4e3ad90`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:38.117596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-833cfe8d-de22-46c9-9108-a038dd881279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:38.220969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 844/1661] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:37 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 09:59:38.222937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4339c59-0f06-41a6-99bd-371e46ca8870 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:38.284189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 818/1662] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:38 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:38.416991 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4339c59-0f06-41a6-99bd-371e46ca8870 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:38.435328 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3649d-188c-4e7c-844b-cd9f2223e9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:38.436719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3649d-188c-4e7c-844b-cd9f2223e9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:38.436719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3649d-188c-4e7c-844b-cd9f2223e9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:38.437058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3649d-188c-4e7c-844b-cd9f2223e9d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:38.438034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3649d-188c-4e7c-844b-cd9f2223e9d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:38.442254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3649d-188c-4e7c-844b-cd9f2223e9d7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:38.576707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 845/1663] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:38 2022] POST /identity/v3/users => generated 292 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:38.581662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cfe8d-de22-46c9-9108-a038dd881279 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:38.598867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cfe8d-de22-46c9-9108-a038dd881279 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:38.613869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55b319-6d72-46a0-a6b0-53d6f9396ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:38.616048 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b319-6d72-46a0-a6b0-53d6f9396ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:38.616347 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b319-6d72-46a0-a6b0-53d6f9396ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:38.618017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b319-6d72-46a0-a6b0-53d6f9396ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:38.618717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55b319-6d72-46a0-a6b0-53d6f9396ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:38.623364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55b319-6d72-46a0-a6b0-53d6f9396ed2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:38.648647 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 819/1664] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:38.652860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3649d-188c-4e7c-844b-cd9f2223e9d7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:38.672952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3649d-188c-4e7c-844b-cd9f2223e9d7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:38.685814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f87126-402d-4cd4-a910-1b58ddc1c09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:38.691862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f87126-402d-4cd4-a910-1b58ddc1c09c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:38.691862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f87126-402d-4cd4-a910-1b58ddc1c09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:38.691862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f87126-402d-4cd4-a910-1b58ddc1c09c admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:38.694395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f87126-402d-4cd4-a910-1b58ddc1c09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:38.704961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f87126-402d-4cd4-a910-1b58ddc1c09c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:38.749072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 846/1665] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:38.753739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55b319-6d72-46a0-a6b0-53d6f9396ed2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:38.768898 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55b319-6d72-46a0-a6b0-53d6f9396ed2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:38.775872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a32baf-1a3d-4800-84c7-9ffd922693b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:38.776806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a32baf-1a3d-4800-84c7-9ffd922693b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:38.777265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a32baf-1a3d-4800-84c7-9ffd922693b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:38.777676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a32baf-1a3d-4800-84c7-9ffd922693b3 admin admin] PATH_INFO: `/v3/projects/fd6b0b0f7672400c95e4dff6c4eec39f/users/856ba5d01b5c4c108df9dc05e4387d12/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:38.833336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a32baf-1a3d-4800-84c7-9ffd922693b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6b0b0f7672400c95e4dff6c4eec39f, user_id=856ba5d01b5c4c108df9dc05e4387d12, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:38.836982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a32baf-1a3d-4800-84c7-9ffd922693b3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:38.934291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 820/1666] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:38 2022] PUT /identity/v3/projects/fd6b0b0f7672400c95e4dff6c4eec39f/users/856ba5d01b5c4c108df9dc05e4387d12/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:38.951478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cbed1-2dc9-4917-8f64-4935cef16081 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:38.959075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cbed1-2dc9-4917-8f64-4935cef16081 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:38.959075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cbed1-2dc9-4917-8f64-4935cef16081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:39.024128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-771cbed1-2dc9-4917-8f64-4935cef16081 None None] MFA Rules not processed for user `856ba5d01b5c4c108df9dc05e4387d12`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:39.030251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-771cbed1-2dc9-4917-8f64-4935cef16081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:39.040473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:39.177931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 847/1667] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:39.182828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a32baf-1a3d-4800-84c7-9ffd922693b3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:39.264813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:39.300202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a32baf-1a3d-4800-84c7-9ffd922693b3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:39.309882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a2c47-4cef-488f-ac09-42f32ad652fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:39.316660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a2c47-4cef-488f-ac09-42f32ad652fa admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:39.317079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a2c47-4cef-488f-ac09-42f32ad652fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:39.318294 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a2c47-4cef-488f-ac09-42f32ad652fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:39.318294 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a2c47-4cef-488f-ac09-42f32ad652fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:39.320610 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a2c47-4cef-488f-ac09-42f32ad652fa admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:39.430896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 821/1668] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:39 2022] POST /identity/v3/projects => generated 383 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:39.435025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771cbed1-2dc9-4917-8f64-4935cef16081 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:39.471214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771cbed1-2dc9-4917-8f64-4935cef16081 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:39.674034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792450f-22f8-4946-ad97-194be3b2d894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:39.674679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792450f-22f8-4946-ad97-194be3b2d894 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:39.675010 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792450f-22f8-4946-ad97-194be3b2d894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:39.675433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792450f-22f8-4946-ad97-194be3b2d894 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:39.676084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792450f-22f8-4946-ad97-194be3b2d894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:39.678475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792450f-22f8-4946-ad97-194be3b2d894 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:39.791338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 848/1669] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:39 2022] POST /identity/v3/users => generated 298 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:39.795660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a2c47-4cef-488f-ac09-42f32ad652fa admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:39.811499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a2c47-4cef-488f-ac09-42f32ad652fa admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:39.839332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb5bb7-59e1-4f0c-90d5-d2dcac881ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:39.840039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb5bb7-59e1-4f0c-90d5-d2dcac881ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:39.840319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb5bb7-59e1-4f0c-90d5-d2dcac881ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:39.842427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb5bb7-59e1-4f0c-90d5-d2dcac881ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:39.843154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb5bb7-59e1-4f0c-90d5-d2dcac881ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:39.853955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb5bb7-59e1-4f0c-90d5-d2dcac881ceb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:39.879404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 822/1670] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:39.883511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792450f-22f8-4946-ad97-194be3b2d894 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:39.898814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792450f-22f8-4946-ad97-194be3b2d894 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:39.916584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d799594-38f5-4ca2-9d20-269dde5cfeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:39.917433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d799594-38f5-4ca2-9d20-269dde5cfeea admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:39.917631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d799594-38f5-4ca2-9d20-269dde5cfeea admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:39.917884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d799594-38f5-4ca2-9d20-269dde5cfeea admin admin] PATH_INFO: `/v3/projects/bb3963aeeed0448daf1a37795ac58562/users/ce5060a128ae488e9885a18d25b990f2/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:39.947218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d799594-38f5-4ca2-9d20-269dde5cfeea admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3963aeeed0448daf1a37795ac58562, user_id=ce5060a128ae488e9885a18d25b990f2, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:39.950039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d799594-38f5-4ca2-9d20-269dde5cfeea admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:40.075752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 849/1671] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:39 2022] PUT /identity/v3/projects/bb3963aeeed0448daf1a37795ac58562/users/ce5060a128ae488e9885a18d25b990f2/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 09:59:40.082710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb5bb7-59e1-4f0c-90d5-d2dcac881ceb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:40.102529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb5bb7-59e1-4f0c-90d5-d2dcac881ceb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:40.244558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc406ab-9a25-44a5-a75b-fb6eb008bcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:40.244558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc406ab-9a25-44a5-a75b-fb6eb008bcd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:40.244558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc406ab-9a25-44a5-a75b-fb6eb008bcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:40.244558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc406ab-9a25-44a5-a75b-fb6eb008bcd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:40.245306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc406ab-9a25-44a5-a75b-fb6eb008bcd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:40.245742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc406ab-9a25-44a5-a75b-fb6eb008bcd8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:40.280023 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 823/1672] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:40.281215 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d799594-38f5-4ca2-9d20-269dde5cfeea admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:40.301388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d799594-38f5-4ca2-9d20-269dde5cfeea admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:40.319865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2daa5-b18a-42ed-9121-7e281127f575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:40.320565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2daa5-b18a-42ed-9121-7e281127f575 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:40.320848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2daa5-b18a-42ed-9121-7e281127f575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:40.321161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2daa5-b18a-42ed-9121-7e281127f575 admin admin] PATH_INFO: `/v3/projects/bb3963aeeed0448daf1a37795ac58562/users/ce5060a128ae488e9885a18d25b990f2/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:40.350381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2daa5-b18a-42ed-9121-7e281127f575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3963aeeed0448daf1a37795ac58562, user_id=ce5060a128ae488e9885a18d25b990f2, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:40.357968 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2daa5-b18a-42ed-9121-7e281127f575 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:40.373139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 850/1673] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:40 2022] PUT /identity/v3/projects/bb3963aeeed0448daf1a37795ac58562/users/ce5060a128ae488e9885a18d25b990f2/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:40.384295 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c61285-5472-4794-8067-cb0e75bdbde4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:40.384897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c61285-5472-4794-8067-cb0e75bdbde4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:40.385096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c61285-5472-4794-8067-cb0e75bdbde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:40.429595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d5c61285-5472-4794-8067-cb0e75bdbde4 None None] MFA Rules not processed for user `ce5060a128ae488e9885a18d25b990f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:40.430952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d5c61285-5472-4794-8067-cb0e75bdbde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:40.511671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdba025-53af-4dc3-a7a6-fd667b64a6d6 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:40.512207 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdba025-53af-4dc3-a7a6-fd667b64a6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:40.513651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdba025-53af-4dc3-a7a6-fd667b64a6d6 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:40.514968 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 851/1674] 10.222.0.17 () {54 vars in 921 bytes} [Thu Aug 11 09:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:59:40.580763 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49d740-dc0b-4f0d-a36a-0cef1db8f8ad None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:40.581396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49d740-dc0b-4f0d-a36a-0cef1db8f8ad None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:40.581677 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49d740-dc0b-4f0d-a36a-0cef1db8f8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:40.607192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 824/1675] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:40.620785 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c61285-5472-4794-8067-cb0e75bdbde4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:40.622754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d5c61285-5472-4794-8067-cb0e75bdbde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:40.631404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-bf49d740-dc0b-4f0d-a36a-0cef1db8f8ad None None] MFA Rules not processed for user `10ce125d46b545499de1fcf7072963e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:40.633514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bf49d740-dc0b-4f0d-a36a-0cef1db8f8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:40.705578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:40.760234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c61285-5472-4794-8067-cb0e75bdbde4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:40.760399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:40.763893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e424a-195a-4178-9c0f-e5d98acd636f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:40.764747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e424a-195a-4178-9c0f-e5d98acd636f service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:40.771121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e424a-195a-4178-9c0f-e5d98acd636f service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:40.771685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e424a-195a-4178-9c0f-e5d98acd636f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:40.781613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 852/1676] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Aug 11 09:59:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:59:40.786340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e424a-195a-4178-9c0f-e5d98acd636f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:40.794328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e424a-195a-4178-9c0f-e5d98acd636f service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:40.843180 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 825/1677] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:40.867167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49d740-dc0b-4f0d-a36a-0cef1db8f8ad None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:40.870962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bf49d740-dc0b-4f0d-a36a-0cef1db8f8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:40.898381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49d740-dc0b-4f0d-a36a-0cef1db8f8ad None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:40.904628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb1caf-8a6c-43d2-aba2-fcf5ab3eba4b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:40.920388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb1caf-8a6c-43d2-aba2-fcf5ab3eba4b service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:40.921205 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb1caf-8a6c-43d2-aba2-fcf5ab3eba4b service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:40.921485 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb1caf-8a6c-43d2-aba2-fcf5ab3eba4b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:40.932461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb1caf-8a6c-43d2-aba2-fcf5ab3eba4b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:40.937787 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb1caf-8a6c-43d2-aba2-fcf5ab3eba4b service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:40.947499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 853/1678] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:41.244783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e424a-195a-4178-9c0f-e5d98acd636f service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:41.247487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-461e424a-195a-4178-9c0f-e5d98acd636f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:41.300764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e424a-195a-4178-9c0f-e5d98acd636f service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:41.314808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44d27c-b89d-468e-9dd1-fbe310572944 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:41.315544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44d27c-b89d-468e-9dd1-fbe310572944 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:41.318720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44d27c-b89d-468e-9dd1-fbe310572944 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:41.318720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44d27c-b89d-468e-9dd1-fbe310572944 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:41.426167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44d27c-b89d-468e-9dd1-fbe310572944 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:41.427932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44d27c-b89d-468e-9dd1-fbe310572944 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:41.441402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 826/1679] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:42.754101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876611a9-9db5-49ac-b8ff-bf2aef61f6db None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:42.754536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876611a9-9db5-49ac-b8ff-bf2aef61f6db None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:42.755037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876611a9-9db5-49ac-b8ff-bf2aef61f6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:42.856196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-876611a9-9db5-49ac-b8ff-bf2aef61f6db None None] MFA Rules not processed for user `d6c2fc568ee74d35bba01c6ce294edeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:42.857487 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-876611a9-9db5-49ac-b8ff-bf2aef61f6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:43.073914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 854/1680] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:43.091779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44d27c-b89d-468e-9dd1-fbe310572944 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:43.094370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7d44d27c-b89d-468e-9dd1-fbe310572944 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:43.347773 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44d27c-b89d-468e-9dd1-fbe310572944 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:43.358256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf8ae3-1e64-4e0c-a447-883346361cb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:43.359045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf8ae3-1e64-4e0c-a447-883346361cb6 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:43.359407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf8ae3-1e64-4e0c-a447-883346361cb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:43.359691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf8ae3-1e64-4e0c-a447-883346361cb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:43.371053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf8ae3-1e64-4e0c-a447-883346361cb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:43.372384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf8ae3-1e64-4e0c-a447-883346361cb6 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:43.389817 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 827/1681] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:43.497333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106928a-20b5-47c3-9329-015132c44c44 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:43.497585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106928a-20b5-47c3-9329-015132c44c44 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:43.501466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106928a-20b5-47c3-9329-015132c44c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:43.590084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5106928a-20b5-47c3-9329-015132c44c44 None None] MFA Rules not processed for user `28b31650305943b498f02c882baa0144`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:43.591436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5106928a-20b5-47c3-9329-015132c44c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:43.759391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 855/1682] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:43.785141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf8ae3-1e64-4e0c-a447-883346361cb6 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:43.817022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf8ae3-1e64-4e0c-a447-883346361cb6 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:43.838018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b74bd-2cd1-4ffc-996d-5642c3729c88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:43.839904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b74bd-2cd1-4ffc-996d-5642c3729c88 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:43.840852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b74bd-2cd1-4ffc-996d-5642c3729c88 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:43.842704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b74bd-2cd1-4ffc-996d-5642c3729c88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:43.880665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b74bd-2cd1-4ffc-996d-5642c3729c88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:43.888439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b74bd-2cd1-4ffc-996d-5642c3729c88 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:43.912917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 828/1683] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:44.164255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106928a-20b5-47c3-9329-015132c44c44 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:44.166006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5106928a-20b5-47c3-9329-015132c44c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:44.287270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2eabd0-a56e-45bc-b779-e530b715c106 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:44.287270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2eabd0-a56e-45bc-b779-e530b715c106 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:44.287270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2eabd0-a56e-45bc-b779-e530b715c106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:44.388830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-db2eabd0-a56e-45bc-b779-e530b715c106 None None] MFA Rules not processed for user `c62f05ea732142c1af1c09dd23e7b9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:44.390172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-db2eabd0-a56e-45bc-b779-e530b715c106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:44.424564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106928a-20b5-47c3-9329-015132c44c44 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:44.445502 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:44.447040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:44.447269 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:44.449380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] PATH_INFO: `/v3/users/3353e6a3822a4b5d8e3fddcf1c5a3beb` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:44.451413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3353e6a3822a4b5d8e3fddcf1c5a3beb)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:44.457682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:44.461956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:44.576136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:44.667594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 829/1684] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:59:44.687016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da807b84-1ca5-44c2-8344-5593c888bdd4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:44.687456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da807b84-1ca5-44c2-8344-5593c888bdd4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:44.687740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da807b84-1ca5-44c2-8344-5593c888bdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:44.784808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-da807b84-1ca5-44c2-8344-5593c888bdd4 None None] MFA Rules not processed for user `c62f05ea732142c1af1c09dd23e7b9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:44.786773 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-da807b84-1ca5-44c2-8344-5593c888bdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:44.802836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3353e6a3822a4b5d8e3fddcf1c5a3beb'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:44.808375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 830/1685] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:44.816363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da807b84-1ca5-44c2-8344-5593c888bdd4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:44.818253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-da807b84-1ca5-44c2-8344-5593c888bdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:44.821371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3353e6a3822a4b5d8e3fddcf1c5a3beb'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:44.871295 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da807b84-1ca5-44c2-8344-5593c888bdd4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:44.919984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be67a55-984c-4688-94b7-fc602023d24f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:44.920762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be67a55-984c-4688-94b7-fc602023d24f service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:44.921138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be67a55-984c-4688-94b7-fc602023d24f service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:44.921416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be67a55-984c-4688-94b7-fc602023d24f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:44.921843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3353e6a3822a4b5d8e3fddcf1c5a3beb'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:44.923612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3353e6a3822a4b5d8e3fddcf1c5a3beb'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:44.944257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3353e6a3822a4b5d8e3fddcf1c5a3beb'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:44.953058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be67a55-984c-4688-94b7-fc602023d24f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:44.961548 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be67a55-984c-4688-94b7-fc602023d24f service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:44.964816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 856/1686] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 09:59:44 2022] DELETE /identity/v3/users/3353e6a3822a4b5d8e3fddcf1c5a3beb => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:44.998139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 831/1687] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:44 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:45.117783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:45.120134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:45.281574 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853b249-be14-46aa-9900-dedf863f00f8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:45.292878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:45.293768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:45.294211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:45.295987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] PATH_INFO: `/v3/users/c562afc432404b25bbd49dee793a8ce8` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:45.299786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c562afc432404b25bbd49dee793a8ce8)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:45.302117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:45.508438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be67a55-984c-4688-94b7-fc602023d24f service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:45.522969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be67a55-984c-4688-94b7-fc602023d24f service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:45.534799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19964aa7-fd57-42f2-88a7-825a206f0685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:45.535709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19964aa7-fd57-42f2-88a7-825a206f0685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:45.536158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19964aa7-fd57-42f2-88a7-825a206f0685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:45.536601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19964aa7-fd57-42f2-88a7-825a206f0685 admin admin] PATH_INFO: `/v3/users/856ba5d01b5c4c108df9dc05e4387d12` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:45.539270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19964aa7-fd57-42f2-88a7-825a206f0685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=856ba5d01b5c4c108df9dc05e4387d12)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:45.543762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19964aa7-fd57-42f2-88a7-825a206f0685 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:45.600188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c562afc432404b25bbd49dee793a8ce8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:45.623318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c562afc432404b25bbd49dee793a8ce8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:45.693429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c562afc432404b25bbd49dee793a8ce8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:45.694599 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c562afc432404b25bbd49dee793a8ce8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:45.724916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c562afc432404b25bbd49dee793a8ce8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:45.729751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:45.745855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 857/1688] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 09:59:45 2022] DELETE /identity/v3/users/c562afc432404b25bbd49dee793a8ce8 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:45.749703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:45.753061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:45.815532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19964aa7-fd57-42f2-88a7-825a206f0685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '856ba5d01b5c4c108df9dc05e4387d12'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:45.821928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19964aa7-fd57-42f2-88a7-825a206f0685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '856ba5d01b5c4c108df9dc05e4387d12'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:45.832871 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19964aa7-fd57-42f2-88a7-825a206f0685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '856ba5d01b5c4c108df9dc05e4387d12'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:45.840314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3174f2d-c46b-41d3-9674-d0a02374280a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:45.849208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:45.854778 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:45.855329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:45.855750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] PATH_INFO: `/v3/projects/f3092f4cfd4e4372aee85c2567de2c5b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:45.857393 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3092f4cfd4e4372aee85c2567de2c5b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:45.861123 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:45.927585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19964aa7-fd57-42f2-88a7-825a206f0685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '856ba5d01b5c4c108df9dc05e4387d12'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:45.952352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19964aa7-fd57-42f2-88a7-825a206f0685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '856ba5d01b5c4c108df9dc05e4387d12'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:45.988224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 832/1689] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 09:59:45 2022] DELETE /identity/v3/users/856ba5d01b5c4c108df9dc05e4387d12 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 09:59:45.989961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467d117-fae9-46d6-9789-350987159646 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:45.992355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467d117-fae9-46d6-9789-350987159646 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:45.993208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467d117-fae9-46d6-9789-350987159646 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:45.996576 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 833/1690] 10.222.0.17 () {54 vars in 912 bytes} [Thu Aug 11 09:59:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:59:46.000110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5306020-afad-4d90-a82f-d9b5b33e998d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:46.005967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5306020-afad-4d90-a82f-d9b5b33e998d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:46.006536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5306020-afad-4d90-a82f-d9b5b33e998d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:46.006920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3092f4cfd4e4372aee85c2567de2c5b', 'request_id': 'req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:46.020683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3092f4cfd4e4372aee85c2567de2c5b', 'request_id': 'req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:46.034478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3092f4cfd4e4372aee85c2567de2c5b', 'request_id': 'req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:46.058299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f5306020-afad-4d90-a82f-d9b5b33e998d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:46.061192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f5306020-afad-4d90-a82f-d9b5b33e998d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:46.112292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] The token cache is being invalidate because project f3092f4cfd4e4372aee85c2567de2c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 09:59:46.112715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:46.113852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-13ffcd9c-c99f-4aa4-be15-48c4c48d4767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:46.120616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 858/1691] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 09:59:45 2022] DELETE /identity/v3/projects/f3092f4cfd4e4372aee85c2567de2c5b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:46.122853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0887f1-cafc-421e-98fc-8f9ddde9fea5 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:46.124427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0887f1-cafc-421e-98fc-8f9ddde9fea5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:46.126432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0887f1-cafc-421e-98fc-8f9ddde9fea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:46.236063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ce0887f1-cafc-421e-98fc-8f9ddde9fea5 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:46.238805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ce0887f1-cafc-421e-98fc-8f9ddde9fea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:46.285706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 834/1692] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:46.289271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6aa9f-e594-4fc5-80ba-b9a63a77b257 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:46.289271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6aa9f-e594-4fc5-80ba-b9a63a77b257 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:46.289271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6aa9f-e594-4fc5-80ba-b9a63a77b257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:46.375222 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 859/1693] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:46.376821 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0887f1-cafc-421e-98fc-8f9ddde9fea5 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:46.378382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ce0887f1-cafc-421e-98fc-8f9ddde9fea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:46.395779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0887f1-cafc-421e-98fc-8f9ddde9fea5 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:46.407076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80044e-586d-48bf-803a-de5ccb04a725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:46.408667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80044e-586d-48bf-803a-de5ccb04a725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:46.409284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80044e-586d-48bf-803a-de5ccb04a725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:46.409669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80044e-586d-48bf-803a-de5ccb04a725 admin admin] PATH_INFO: `/v3/users/089508b9ca2f42209f6ae189e4e3ad90` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:46.412848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80044e-586d-48bf-803a-de5ccb04a725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=089508b9ca2f42209f6ae189e4e3ad90)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:46.414983 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80044e-586d-48bf-803a-de5ccb04a725 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:46.434633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-79a6aa9f-e594-4fc5-80ba-b9a63a77b257 None None] MFA Rules not processed for user `4fc6c3d1ce8645808c22d1b59b15a100`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:46.435985 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-79a6aa9f-e594-4fc5-80ba-b9a63a77b257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:46.463303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:46.554583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 835/1694] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 09:59:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:46.556399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6aa9f-e594-4fc5-80ba-b9a63a77b257 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:46.558113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-79a6aa9f-e594-4fc5-80ba-b9a63a77b257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:46.572351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6aa9f-e594-4fc5-80ba-b9a63a77b257 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:46.582518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca486c-16f9-486b-b348-3b7383101848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:46.583505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca486c-16f9-486b-b348-3b7383101848 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:46.583987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca486c-16f9-486b-b348-3b7383101848 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:46.584824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca486c-16f9-486b-b348-3b7383101848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:46.619222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca486c-16f9-486b-b348-3b7383101848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:46.622021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca486c-16f9-486b-b348-3b7383101848 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:46.637190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 836/1695] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:46.638610 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca486c-16f9-486b-b348-3b7383101848 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:46.653485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca486c-16f9-486b-b348-3b7383101848 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:46.664541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a216072-71d1-45f0-ac2b-dceac5aa197c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:46.672839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a216072-71d1-45f0-ac2b-dceac5aa197c service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:46.672839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a216072-71d1-45f0-ac2b-dceac5aa197c service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:46.672839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a216072-71d1-45f0-ac2b-dceac5aa197c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:46.679729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f80044e-586d-48bf-803a-de5ccb04a725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '089508b9ca2f42209f6ae189e4e3ad90'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:46.688658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a216072-71d1-45f0-ac2b-dceac5aa197c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:46.690672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a216072-71d1-45f0-ac2b-dceac5aa197c service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:46.716193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 837/1696] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:46.716961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a216072-71d1-45f0-ac2b-dceac5aa197c service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:46.721251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f80044e-586d-48bf-803a-de5ccb04a725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '089508b9ca2f42209f6ae189e4e3ad90'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:46.740721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a216072-71d1-45f0-ac2b-dceac5aa197c service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:46.753833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92a2c9-9542-40d7-848c-1e7bdfa0aed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:46.756323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92a2c9-9542-40d7-848c-1e7bdfa0aed1 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:46.756626 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92a2c9-9542-40d7-848c-1e7bdfa0aed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:46.758203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92a2c9-9542-40d7-848c-1e7bdfa0aed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:46.779997 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92a2c9-9542-40d7-848c-1e7bdfa0aed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:46.784109 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92a2c9-9542-40d7-848c-1e7bdfa0aed1 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:46.796029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 838/1697] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:46.813667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f80044e-586d-48bf-803a-de5ccb04a725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '089508b9ca2f42209f6ae189e4e3ad90'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:46.817059 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f80044e-586d-48bf-803a-de5ccb04a725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '089508b9ca2f42209f6ae189e4e3ad90'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:46.854062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f80044e-586d-48bf-803a-de5ccb04a725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '089508b9ca2f42209f6ae189e4e3ad90'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:46.867180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 860/1698] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 09:59:46 2022] DELETE /identity/v3/users/089508b9ca2f42209f6ae189e4e3ad90 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:47.339723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339f9e-1317-400d-9def-e8c0c05de609 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:47.340079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339f9e-1317-400d-9def-e8c0c05de609 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:47.340365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da339f9e-1317-400d-9def-e8c0c05de609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:47.424390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-da339f9e-1317-400d-9def-e8c0c05de609 None None] MFA Rules not processed for user `bf9353bc130d45a99af9386ff9dd9a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:47.427748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-da339f9e-1317-400d-9def-e8c0c05de609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:47.601652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 839/1699] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:47.616060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:47.616880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2dede4-5d3e-4ffe-9459-4f9684785ab5 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:47.617689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2dede4-5d3e-4ffe-9459-4f9684785ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:47.619589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2dede4-5d3e-4ffe-9459-4f9684785ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:47.683258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ce2dede4-5d3e-4ffe-9459-4f9684785ab5 None None] MFA Rules not processed for user `bf9353bc130d45a99af9386ff9dd9a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:47.684023 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ce2dede4-5d3e-4ffe-9459-4f9684785ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:47.728707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 861/1700] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:47.739777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da339f9e-1317-400d-9def-e8c0c05de609 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:47.741197 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-da339f9e-1317-400d-9def-e8c0c05de609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:47.864772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da339f9e-1317-400d-9def-e8c0c05de609 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:47.872577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df6008-abf2-42bd-86e6-5b3ffc30f6b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:47.873819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df6008-abf2-42bd-86e6-5b3ffc30f6b0 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:47.878929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df6008-abf2-42bd-86e6-5b3ffc30f6b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:47.878929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df6008-abf2-42bd-86e6-5b3ffc30f6b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:47.887140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df6008-abf2-42bd-86e6-5b3ffc30f6b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:47.889565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df6008-abf2-42bd-86e6-5b3ffc30f6b0 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:47.904867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 840/1701] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:47.913393 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2dede4-5d3e-4ffe-9459-4f9684785ab5 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:47.914785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ce2dede4-5d3e-4ffe-9459-4f9684785ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:48.098367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2dede4-5d3e-4ffe-9459-4f9684785ab5 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:48.116532 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:48.117308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:48.117599 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:48.117872 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] PATH_INFO: `/v3/projects/b0295d2b57434ce197af9875347cfb76` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:48.119068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0295d2b57434ce197af9875347cfb76)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:48.121597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:48.386539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0295d2b57434ce197af9875347cfb76', 'request_id': 'req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:48.393411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0295d2b57434ce197af9875347cfb76', 'request_id': 'req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:48.410110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0295d2b57434ce197af9875347cfb76', 'request_id': 'req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:48.657250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df6008-abf2-42bd-86e6-5b3ffc30f6b0 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:48.660041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-99df6008-abf2-42bd-86e6-5b3ffc30f6b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:48.663725 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] The token cache is being invalidate because project b0295d2b57434ce197af9875347cfb76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 09:59:48.663946 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:48.665942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-92ebd5de-bc54-4bdb-9efb-e1cc9a04d183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:48.669460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 862/1702] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 09:59:47 2022] DELETE /identity/v3/projects/b0295d2b57434ce197af9875347cfb76 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:48.681048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83eef9-8252-4d48-8212-89c10e2a7fea None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:48.681333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83eef9-8252-4d48-8212-89c10e2a7fea None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:48.681610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83eef9-8252-4d48-8212-89c10e2a7fea None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:48.683116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 863/1703] 10.222.0.17 () {54 vars in 912 bytes} [Thu Aug 11 09:59:48 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:59:48.691417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:48.697799 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83eef9-8252-4d48-8212-89c10e2a7fea None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:48.707333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df6008-abf2-42bd-86e6-5b3ffc30f6b0 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:48.712589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4e83eef9-8252-4d48-8212-89c10e2a7fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:48.863390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:48.864305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:48.864743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:48.865056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] PATH_INFO: `/v3/projects/33671f24dbc4478682f5e7b090bf48fa` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:48.868456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33671f24dbc4478682f5e7b090bf48fa)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:48.869529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 09:59:48.869529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 11 09:59:48.869529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: Aug 11 09:59:48.871976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:48.912588 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83eef9-8252-4d48-8212-89c10e2a7fea None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:48.922631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:48.923324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:48.923556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:48.923992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] PATH_INFO: `/v3/users/7cb12d703dc148688958302e869eb71f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:48.929347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb12d703dc148688958302e869eb71f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:48.934171 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:49.041861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33671f24dbc4478682f5e7b090bf48fa', 'request_id': 'req-ac63d635-059c-4690-9179-871b8ca02250'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.051627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33671f24dbc4478682f5e7b090bf48fa', 'request_id': 'req-ac63d635-059c-4690-9179-871b8ca02250'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.070622 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33671f24dbc4478682f5e7b090bf48fa', 'request_id': 'req-ac63d635-059c-4690-9179-871b8ca02250'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.169255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] The token cache is being invalidate because project 33671f24dbc4478682f5e7b090bf48fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 09:59:49.169255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.170042 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ac63d635-059c-4690-9179-871b8ca02250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.171863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 841/1704] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 09:59:48 2022] DELETE /identity/v3/projects/33671f24dbc4478682f5e7b090bf48fa => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 09:59:49.173476 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de3cfb-faa1-428e-ab86-64115002f866 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:49.173828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de3cfb-faa1-428e-ab86-64115002f866 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:49.174187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de3cfb-faa1-428e-ab86-64115002f866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:49.211539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-33de3cfb-faa1-428e-ab86-64115002f866 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:49.213204 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-33de3cfb-faa1-428e-ab86-64115002f866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:49.217543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb12d703dc148688958302e869eb71f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.226506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb12d703dc148688958302e869eb71f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.317569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb12d703dc148688958302e869eb71f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.319140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb12d703dc148688958302e869eb71f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.327109 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 842/1705] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:49.329957 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e104760-2536-4861-a605-72ab7489a05e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:49.330461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e104760-2536-4861-a605-72ab7489a05e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:49.331497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e104760-2536-4861-a605-72ab7489a05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:49.357837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb12d703dc148688958302e869eb71f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.367885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1e104760-2536-4861-a605-72ab7489a05e None None] MFA Rules not processed for user `4fc6c3d1ce8645808c22d1b59b15a100`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:49.369756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1e104760-2536-4861-a605-72ab7489a05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:49.382592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 864/1706] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 09:59:48 2022] DELETE /identity/v3/users/7cb12d703dc148688958302e869eb71f => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:49.386255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:49.390000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:49.416834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:49.444988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 843/1707] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:49.446526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57eeb7-d048-4207-8d45-40b01d8c56f8 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:49.447023 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57eeb7-d048-4207-8d45-40b01d8c56f8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:49.447300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57eeb7-d048-4207-8d45-40b01d8c56f8 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:49.449007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 844/1708] 10.222.0.17 () {54 vars in 961 bytes} [Thu Aug 11 09:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 09:59:49.450259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506a596-74a3-47c8-ae87-d3a41288dde0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:49.450667 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506a596-74a3-47c8-ae87-d3a41288dde0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:49.451020 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506a596-74a3-47c8-ae87-d3a41288dde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:49.462625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:49.497996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3506a596-74a3-47c8-ae87-d3a41288dde0 None None] MFA Rules not processed for user `c62f05ea732142c1af1c09dd23e7b9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:49.499379 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3506a596-74a3-47c8-ae87-d3a41288dde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:49.508333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cc9cb-e22c-4d30-809f-959b9a4a23b0 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:49.513618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:49.518756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:49.518756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:49.518756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] PATH_INFO: `/v3/projects/fd6b0b0f7672400c95e4dff6c4eec39f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:49.518756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd6b0b0f7672400c95e4dff6c4eec39f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:49.519471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:49.630521 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 845/1709] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:49.631826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aae43c-53a9-4365-9bbc-0820ecf5f314 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:49.633542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aae43c-53a9-4365-9bbc-0820ecf5f314 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:49.633542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aae43c-53a9-4365-9bbc-0820ecf5f314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:49.669844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-98aae43c-53a9-4365-9bbc-0820ecf5f314 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:49.671233 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-98aae43c-53a9-4365-9bbc-0820ecf5f314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:49.685198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 846/1710] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:49.690940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07653c-6c43-4972-a2a9-5786ce87bd52 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:49.690940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07653c-6c43-4972-a2a9-5786ce87bd52 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:49.690940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07653c-6c43-4972-a2a9-5786ce87bd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:49.767314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd6b0b0f7672400c95e4dff6c4eec39f', 'request_id': 'req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.772712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8d07653c-6c43-4972-a2a9-5786ce87bd52 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:49.774015 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8d07653c-6c43-4972-a2a9-5786ce87bd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:49.801592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd6b0b0f7672400c95e4dff6c4eec39f', 'request_id': 'req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.827007 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd6b0b0f7672400c95e4dff6c4eec39f', 'request_id': 'req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.931461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] The token cache is being invalidate because project fd6b0b0f7672400c95e4dff6c4eec39f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 09:59:49.932543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.933433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e9c98f8c-a5f4-4a8c-86bb-a1b4cebac8f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:49.935409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 865/1711] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 09:59:49 2022] DELETE /identity/v3/projects/fd6b0b0f7672400c95e4dff6c4eec39f => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:49.936625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4bbfc-2372-4d64-8ee2-41e25bfc57da None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:49.936903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4bbfc-2372-4d64-8ee2-41e25bfc57da None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:49.937229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4bbfc-2372-4d64-8ee2-41e25bfc57da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:49.995885 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b4c4bbfc-2372-4d64-8ee2-41e25bfc57da None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:49.997061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 847/1712] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:50.001326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b4c4bbfc-2372-4d64-8ee2-41e25bfc57da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:50.002082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55a425-be90-4ab0-a48c-4ff7345f4798 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:50.003280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55a425-be90-4ab0-a48c-4ff7345f4798 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:50.004627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55a425-be90-4ab0-a48c-4ff7345f4798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:50.010691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 866/1713] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:50.012231 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4bbfc-2372-4d64-8ee2-41e25bfc57da None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:50.013816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b4c4bbfc-2372-4d64-8ee2-41e25bfc57da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:50.055535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ca55a425-be90-4ab0-a48c-4ff7345f4798 None None] MFA Rules not processed for user `10ce125d46b545499de1fcf7072963e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:50.059386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ca55a425-be90-4ab0-a48c-4ff7345f4798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:50.224188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4bbfc-2372-4d64-8ee2-41e25bfc57da None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:50.244227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bbb3d-8e81-4af9-8bb1-178397b349d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:50.245334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bbb3d-8e81-4af9-8bb1-178397b349d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:50.245926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bbb3d-8e81-4af9-8bb1-178397b349d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:50.246436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bbb3d-8e81-4af9-8bb1-178397b349d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:50.249227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e77bbb3d-8e81-4af9-8bb1-178397b349d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:50.258677 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 848/1714] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Aug 11 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:50.260239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d4373-d4a2-430a-afc2-4a27f72328fc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:50.260562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d4373-d4a2-430a-afc2-4a27f72328fc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:50.260844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d4373-d4a2-430a-afc2-4a27f72328fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:50.292871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bbb3d-8e81-4af9-8bb1-178397b349d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:50.294895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77bbb3d-8e81-4af9-8bb1-178397b349d5 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:50.299294 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4f6d4373-d4a2-430a-afc2-4a27f72328fc None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:50.301122 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4f6d4373-d4a2-430a-afc2-4a27f72328fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:50.311570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 867/1715] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:50.313280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bbb3d-8e81-4af9-8bb1-178397b349d5 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:50.315449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e77bbb3d-8e81-4af9-8bb1-178397b349d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:50.316978 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 849/1716] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:50.318203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d4373-d4a2-430a-afc2-4a27f72328fc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:50.333499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d4373-d4a2-430a-afc2-4a27f72328fc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:50.337643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77bbb3d-8e81-4af9-8bb1-178397b349d5 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:50.343391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:50.351435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef07993-7aa9-4b19-9ec2-22f0b897d2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:50.351601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:50.351601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:50.351814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef07993-7aa9-4b19-9ec2-22f0b897d2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:50.351814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef07993-7aa9-4b19-9ec2-22f0b897d2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:50.351814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef07993-7aa9-4b19-9ec2-22f0b897d2d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:50.352066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] PATH_INFO: `/v3/users/ce5060a128ae488e9885a18d25b990f2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:50.352147 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef07993-7aa9-4b19-9ec2-22f0b897d2d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:50.352147 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef07993-7aa9-4b19-9ec2-22f0b897d2d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:50.352311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5060a128ae488e9885a18d25b990f2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:50.352396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef07993-7aa9-4b19-9ec2-22f0b897d2d0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:50.352486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:50.381988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 850/1717] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:50.382525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef07993-7aa9-4b19-9ec2-22f0b897d2d0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:50.385656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-eef07993-7aa9-4b19-9ec2-22f0b897d2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:50.403997 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:50.406053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef07993-7aa9-4b19-9ec2-22f0b897d2d0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:50.411616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd3da2-fbb0-4c1a-a5a2-c808ad5b3f32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:50.412973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd3da2-fbb0-4c1a-a5a2-c808ad5b3f32 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:50.413657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd3da2-fbb0-4c1a-a5a2-c808ad5b3f32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:50.414277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd3da2-fbb0-4c1a-a5a2-c808ad5b3f32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:50.427492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd3da2-fbb0-4c1a-a5a2-c808ad5b3f32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:50.432572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd3da2-fbb0-4c1a-a5a2-c808ad5b3f32 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:50.442463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 851/1718] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:50.444879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32369f23-ddd5-4d9e-a873-642043983407 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:50.445430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32369f23-ddd5-4d9e-a873-642043983407 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:50.446102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32369f23-ddd5-4d9e-a873-642043983407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:50.474804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-32369f23-ddd5-4d9e-a873-642043983407 None None] MFA Rules not processed for user `d6c2fc568ee74d35bba01c6ce294edeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:50.477212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-32369f23-ddd5-4d9e-a873-642043983407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:50.560101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5060a128ae488e9885a18d25b990f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:50.573333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5060a128ae488e9885a18d25b990f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:50.581090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 852/1719] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:50.587586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32369f23-ddd5-4d9e-a873-642043983407 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:50.624471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:50.640565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32369f23-ddd5-4d9e-a873-642043983407 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:50.653512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5060a128ae488e9885a18d25b990f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:50.654405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5060a128ae488e9885a18d25b990f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:50.665183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cff36-498e-49ce-be23-d0a4f6ba7813 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:50.667136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cff36-498e-49ce-be23-d0a4f6ba7813 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:50.667423 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cff36-498e-49ce-be23-d0a4f6ba7813 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:50.667708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cff36-498e-49ce-be23-d0a4f6ba7813 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:50.674949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4d2cff36-498e-49ce-be23-d0a4f6ba7813 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:50.691631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5060a128ae488e9885a18d25b990f2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:50.706054 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cff36-498e-49ce-be23-d0a4f6ba7813 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:50.707444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 868/1720] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 09:59:50 2022] DELETE /identity/v3/users/ce5060a128ae488e9885a18d25b990f2 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:50.708760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:50.720514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:50.720879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cff36-498e-49ce-be23-d0a4f6ba7813 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:50.762122 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 853/1721] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:50.764463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cff36-498e-49ce-be23-d0a4f6ba7813 service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:50.772815 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4d2cff36-498e-49ce-be23-d0a4f6ba7813 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:50.817827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cff36-498e-49ce-be23-d0a4f6ba7813 service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:50.957580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9afa6c-4960-4f6a-a551-f4a7281965f2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:50.976455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6a15a-c50b-4fd8-9369-c8b59a92b3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:50.977117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6a15a-c50b-4fd8-9369-c8b59a92b3bc admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:50.982444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6a15a-c50b-4fd8-9369-c8b59a92b3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:50.983001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6a15a-c50b-4fd8-9369-c8b59a92b3bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:50.985946 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6a15a-c50b-4fd8-9369-c8b59a92b3bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:50.989486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6a15a-c50b-4fd8-9369-c8b59a92b3bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:50.997824 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6a15a-c50b-4fd8-9369-c8b59a92b3bc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:51.046716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 869/1722] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:51.046716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757801a6-888b-491a-9817-d9af64883e0e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:51.051706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757801a6-888b-491a-9817-d9af64883e0e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:51.052000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757801a6-888b-491a-9817-d9af64883e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:51.088658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c93601-1af6-4969-8018-5e7fcb5d59de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:51.090200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93601-1af6-4969-8018-5e7fcb5d59de admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:51.092792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93601-1af6-4969-8018-5e7fcb5d59de admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:51.094700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93601-1af6-4969-8018-5e7fcb5d59de admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:51.097825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c93601-1af6-4969-8018-5e7fcb5d59de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:51.101731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c93601-1af6-4969-8018-5e7fcb5d59de admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:51.134108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-757801a6-888b-491a-9817-d9af64883e0e None None] MFA Rules not processed for user `28b31650305943b498f02c882baa0144`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:51.139253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-757801a6-888b-491a-9817-d9af64883e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:51.307228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 854/1723] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:50 2022] POST /identity/v3/projects => generated 371 bytes in 545 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:51.316274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c93601-1af6-4969-8018-5e7fcb5d59de admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:51.371974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 870/1724] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:51.373131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757801a6-888b-491a-9817-d9af64883e0e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:51.397978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757801a6-888b-491a-9817-d9af64883e0e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:51.512669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c93601-1af6-4969-8018-5e7fcb5d59de admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:51.519434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0955c-ff53-4619-ac50-eb99750eb8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:51.520890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0955c-ff53-4619-ac50-eb99750eb8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:51.521276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0955c-ff53-4619-ac50-eb99750eb8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:51.521622 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0955c-ff53-4619-ac50-eb99750eb8c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:51.522481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0955c-ff53-4619-ac50-eb99750eb8c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:51.524928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0955c-ff53-4619-ac50-eb99750eb8c1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:51.536825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:51.553484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60680709-6aaa-42ea-a781-018025716873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:51.555236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680709-6aaa-42ea-a781-018025716873 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:51.555698 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680709-6aaa-42ea-a781-018025716873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:51.556422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680709-6aaa-42ea-a781-018025716873 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:51.557311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60680709-6aaa-42ea-a781-018025716873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:51.560172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60680709-6aaa-42ea-a781-018025716873 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:51.633718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 855/1725] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:51 2022] POST /identity/v3/projects => generated 365 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:51.634862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0955c-ff53-4619-ac50-eb99750eb8c1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:51.636745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6fc0955c-ff53-4619-ac50-eb99750eb8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:51.671598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 871/1726] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:51 2022] POST /identity/v3/users => generated 286 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:51.674126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60680709-6aaa-42ea-a781-018025716873 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:51.713617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60680709-6aaa-42ea-a781-018025716873 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:51.736731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0955c-ff53-4619-ac50-eb99750eb8c1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:51.742835 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:51.743443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:51.743820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:51.744105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] PATH_INFO: `/v3/projects/878ea5b76a9d4139bd0a3461d12b3ab0` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:51.745337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=878ea5b76a9d4139bd0a3461d12b3ab0)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:51.747316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:51.802114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:51.882534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f193ce1-0e3d-4c9a-9df8-b6e98d805da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:51.884828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193ce1-0e3d-4c9a-9df8-b6e98d805da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:51.885151 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193ce1-0e3d-4c9a-9df8-b6e98d805da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:51.885491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193ce1-0e3d-4c9a-9df8-b6e98d805da8 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:51.886125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f193ce1-0e3d-4c9a-9df8-b6e98d805da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:51.891777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f193ce1-0e3d-4c9a-9df8-b6e98d805da8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:51.907014 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '878ea5b76a9d4139bd0a3461d12b3ab0', 'request_id': 'req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:51.924391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '878ea5b76a9d4139bd0a3461d12b3ab0', 'request_id': 'req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:51.936458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '878ea5b76a9d4139bd0a3461d12b3ab0', 'request_id': 'req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:52.035618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 872/1727] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:51 2022] POST /identity/v3/users => generated 283 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:52.042384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f193ce1-0e3d-4c9a-9df8-b6e98d805da8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:52.084838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] The token cache is being invalidate because project 878ea5b76a9d4139bd0a3461d12b3ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 09:59:52.085518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:52.086435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:52.091663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f193ce1-0e3d-4c9a-9df8-b6e98d805da8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:52.096618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 856/1728] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 09:59:51 2022] DELETE /identity/v3/projects/878ea5b76a9d4139bd0a3461d12b3ab0 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:52.100589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:52.103169 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:52.276266 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3514e-995f-4fb6-a891-b93857ff0a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:52.277744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3514e-995f-4fb6-a891-b93857ff0a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:52.280229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3514e-995f-4fb6-a891-b93857ff0a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:52.280631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3514e-995f-4fb6-a891-b93857ff0a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:52.281513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3514e-995f-4fb6-a891-b93857ff0a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:52.286245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3514e-995f-4fb6-a891-b93857ff0a6b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:52.298305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495d31f-f5b6-4ddd-9856-fa3efb31b1fb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:52.309488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5821e0c-a7cd-44ae-bed3-f2c65cf18be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:52.313957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 873/1729] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:52.313957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3514e-995f-4fb6-a891-b93857ff0a6b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:52.314194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5821e0c-a7cd-44ae-bed3-f2c65cf18be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:52.314194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5821e0c-a7cd-44ae-bed3-f2c65cf18be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:52.314537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5821e0c-a7cd-44ae-bed3-f2c65cf18be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:52.315578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5821e0c-a7cd-44ae-bed3-f2c65cf18be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:52.318587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c9b3514e-995f-4fb6-a891-b93857ff0a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:52.320817 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5821e0c-a7cd-44ae-bed3-f2c65cf18be6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:52.347705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3514e-995f-4fb6-a891-b93857ff0a6b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:52.348221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 857/1730] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:52.348966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5821e0c-a7cd-44ae-bed3-f2c65cf18be6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:52.353659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b5821e0c-a7cd-44ae-bed3-f2c65cf18be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:52.357925 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:52.359190 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:52.360279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:52.360650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] PATH_INFO: `/v3/users/8e1b09cd8e4342079a0b173035ee505b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:52.362298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e1b09cd8e4342079a0b173035ee505b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:52.364545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:52.371582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5821e0c-a7cd-44ae-bed3-f2c65cf18be6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:52.377572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93689974-c4ce-4349-bfff-c4a2f4131f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:52.378224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93689974-c4ce-4349-bfff-c4a2f4131f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:52.378541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93689974-c4ce-4349-bfff-c4a2f4131f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:52.378816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93689974-c4ce-4349-bfff-c4a2f4131f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:52.379805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93689974-c4ce-4349-bfff-c4a2f4131f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:52.381340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93689974-c4ce-4349-bfff-c4a2f4131f17 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:52.400198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 858/1731] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:52.400198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93689974-c4ce-4349-bfff-c4a2f4131f17 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:52.408262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93689974-c4ce-4349-bfff-c4a2f4131f17 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:52.415005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102716d-5f52-46c2-8f05-493186f0c76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:52.415684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102716d-5f52-46c2-8f05-493186f0c76f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:52.415973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102716d-5f52-46c2-8f05-493186f0c76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:52.416257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102716d-5f52-46c2-8f05-493186f0c76f admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:52.417132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102716d-5f52-46c2-8f05-493186f0c76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:52.418711 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102716d-5f52-46c2-8f05-493186f0c76f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:52.436093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 859/1732] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:52.436093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102716d-5f52-46c2-8f05-493186f0c76f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:52.450117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102716d-5f52-46c2-8f05-493186f0c76f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:52.484127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f7094-61ca-46ea-960d-3990b59db1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:52.485052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f7094-61ca-46ea-960d-3990b59db1dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:52.485346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f7094-61ca-46ea-960d-3990b59db1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:52.485873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f7094-61ca-46ea-960d-3990b59db1dc admin admin] PATH_INFO: `/v3/projects/4ed0ea5ffa3e453689ce34ea857cd47a/users/99d2a1ac104a49cbb52da25742f17ec9/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:52.520166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f7094-61ca-46ea-960d-3990b59db1dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed0ea5ffa3e453689ce34ea857cd47a, user_id=99d2a1ac104a49cbb52da25742f17ec9, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:52.522787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f7094-61ca-46ea-960d-3990b59db1dc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:52.643295 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:52.653472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 860/1733] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:52 2022] PUT /identity/v3/projects/4ed0ea5ffa3e453689ce34ea857cd47a/users/99d2a1ac104a49cbb52da25742f17ec9/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:52.654607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f7094-61ca-46ea-960d-3990b59db1dc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:52.667244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e1b09cd8e4342079a0b173035ee505b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:52.680221 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e1b09cd8e4342079a0b173035ee505b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:52.795024 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e1b09cd8e4342079a0b173035ee505b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:52.796066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f7094-61ca-46ea-960d-3990b59db1dc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:52.799521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e1b09cd8e4342079a0b173035ee505b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:52.803521 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968a58b-fae8-4ba5-a7c3-0b0f429e7e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:52.804858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968a58b-fae8-4ba5-a7c3-0b0f429e7e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:52.804858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968a58b-fae8-4ba5-a7c3-0b0f429e7e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:52.805112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968a58b-fae8-4ba5-a7c3-0b0f429e7e9d admin admin] PATH_INFO: `/v3/projects/66d476cb3df149d48a0186e7aaf85808/users/1688e9a113a746889abf85084d4ec309/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:52.822297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968a58b-fae8-4ba5-a7c3-0b0f429e7e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=66d476cb3df149d48a0186e7aaf85808, user_id=1688e9a113a746889abf85084d4ec309, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:52.825541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968a58b-fae8-4ba5-a7c3-0b0f429e7e9d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:52.834295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e1b09cd8e4342079a0b173035ee505b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:52.850002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 874/1734] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 09:59:52 2022] DELETE /identity/v3/users/8e1b09cd8e4342079a0b173035ee505b => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:52.856688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:52.860925 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:52.969814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 861/1735] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:52 2022] PUT /identity/v3/projects/66d476cb3df149d48a0186e7aaf85808/users/1688e9a113a746889abf85084d4ec309/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:52.973341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31da12-87a4-4438-b57f-fb6d481e72ed None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:52.974196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31da12-87a4-4438-b57f-fb6d481e72ed None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:52.975007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31da12-87a4-4438-b57f-fb6d481e72ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:53.013303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac97294-8841-443f-af13-205236a4c9f4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:53.072103 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5c31da12-87a4-4438-b57f-fb6d481e72ed None None] MFA Rules not processed for user `99d2a1ac104a49cbb52da25742f17ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:53.079783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5c31da12-87a4-4438-b57f-fb6d481e72ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:53.095492 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:53.097001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:53.102402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:53.102402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] PATH_INFO: `/v3/projects/bb3963aeeed0448daf1a37795ac58562` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:53.102402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb3963aeeed0448daf1a37795ac58562)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:53.103747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:53.237589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 862/1736] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:53.240096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b46c-e87c-4490-87d3-a2a6651f125e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:53.240096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b46c-e87c-4490-87d3-a2a6651f125e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:53.240266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2b46c-e87c-4490-87d3-a2a6651f125e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:53.257784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb3963aeeed0448daf1a37795ac58562', 'request_id': 'req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:53.268861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb3963aeeed0448daf1a37795ac58562', 'request_id': 'req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:53.273291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb3963aeeed0448daf1a37795ac58562', 'request_id': 'req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:53.289730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:53.322216 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-85d2b46c-e87c-4490-87d3-a2a6651f125e None None] MFA Rules not processed for user `1688e9a113a746889abf85084d4ec309`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:53.323500 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-85d2b46c-e87c-4490-87d3-a2a6651f125e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:53.387205 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] The token cache is being invalidate because project bb3963aeeed0448daf1a37795ac58562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 09:59:53.390343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:53.390343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:53.390343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 875/1737] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 09:59:52 2022] DELETE /identity/v3/projects/bb3963aeeed0448daf1a37795ac58562 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:53.391674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:53.393330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:53.501359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 863/1738] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:53.501359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830aee2-d9c2-4256-97e2-38b21ad6407a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:53.501359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830aee2-d9c2-4256-97e2-38b21ad6407a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:53.501359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830aee2-d9c2-4256-97e2-38b21ad6407a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:53.591758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a830aee2-d9c2-4256-97e2-38b21ad6407a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:53.591758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a830aee2-d9c2-4256-97e2-38b21ad6407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:53.610430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba5f3e-7a7a-4a5d-97c0-2684e692dc92 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:53.621307 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e556090-4e7f-44ad-bc57-6f115f53e53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:53.623791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e556090-4e7f-44ad-bc57-6f115f53e53a admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:53.625855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e556090-4e7f-44ad-bc57-6f115f53e53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:53.626268 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 864/1739] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:53.627263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e556090-4e7f-44ad-bc57-6f115f53e53a admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:53.627378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830aee2-d9c2-4256-97e2-38b21ad6407a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:53.629296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e556090-4e7f-44ad-bc57-6f115f53e53a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:53.634032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a830aee2-d9c2-4256-97e2-38b21ad6407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:53.637717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e556090-4e7f-44ad-bc57-6f115f53e53a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:53.649445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830aee2-d9c2-4256-97e2-38b21ad6407a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:53.655192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:53.656121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:53.657275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:53.657904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] PATH_INFO: `/v3/projects/91fc5be35c9f4a7283162c7565e04259` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:53.659587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] RBAC: Authorizing `identity:delete_project(project_id=91fc5be35c9f4a7283162c7565e04259)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:53.661725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:53.808409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 876/1740] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:53 2022] POST /identity/v3/projects => generated 373 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:53.812161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e556090-4e7f-44ad-bc57-6f115f53e53a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:53.819052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0e556090-4e7f-44ad-bc57-6f115f53e53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:53.841843 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91fc5be35c9f4a7283162c7565e04259', 'request_id': 'req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:53.873801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91fc5be35c9f4a7283162c7565e04259', 'request_id': 'req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:53.877909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:53.900937 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91fc5be35c9f4a7283162c7565e04259', 'request_id': 'req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:53.965933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e556090-4e7f-44ad-bc57-6f115f53e53a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:53.971682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4fbf9-ff85-456a-b398-1c903dd05439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:53.972455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4fbf9-ff85-456a-b398-1c903dd05439 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:53.972982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4fbf9-ff85-456a-b398-1c903dd05439 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:53.973402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4fbf9-ff85-456a-b398-1c903dd05439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:53.997216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:53.998725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] The token cache is being invalidate because project 91fc5be35c9f4a7283162c7565e04259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 09:59:53.999241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:54.002599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e0aaba3c-9a6a-46d7-818c-a6dd46e4367e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 09:59:54.004961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 865/1741] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 09:59:53 2022] DELETE /identity/v3/projects/91fc5be35c9f4a7283162c7565e04259 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:54.006845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48feaa-c6db-4456-ab7e-0c6f5cb6a7f7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:54.007473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48feaa-c6db-4456-ab7e-0c6f5cb6a7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:54.010724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48feaa-c6db-4456-ab7e-0c6f5cb6a7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:54.056585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2b48feaa-c6db-4456-ab7e-0c6f5cb6a7f7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:54.058118 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2b48feaa-c6db-4456-ab7e-0c6f5cb6a7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:54.111102 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4fbf9-ff85-456a-b398-1c903dd05439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:54.112614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4fbf9-ff85-456a-b398-1c903dd05439 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:54.125998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 877/1742] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:54.127071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4fbf9-ff85-456a-b398-1c903dd05439 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:54.237783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 866/1743] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:54.240546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad17d-d5ee-441f-9772-9988555544b4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:54.246497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad17d-d5ee-441f-9772-9988555544b4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:54.246497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad17d-d5ee-441f-9772-9988555544b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:54.246777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4fbf9-ff85-456a-b398-1c903dd05439 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:54.251028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6c022-060d-4a98-8a7c-412b068f2320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:54.251903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6c022-060d-4a98-8a7c-412b068f2320 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:54.252329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6c022-060d-4a98-8a7c-412b068f2320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:54.252764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6c022-060d-4a98-8a7c-412b068f2320 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:54.253462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6c022-060d-4a98-8a7c-412b068f2320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:54.255598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6c022-060d-4a98-8a7c-412b068f2320 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:54.278458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ed0ad17d-d5ee-441f-9772-9988555544b4 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:54.279981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ed0ad17d-d5ee-441f-9772-9988555544b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:54.289751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 867/1744] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:54.290563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ad17d-d5ee-441f-9772-9988555544b4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:54.312153 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ad17d-d5ee-441f-9772-9988555544b4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:54.322501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120038a-c848-4549-9ec4-7e3cb4c36cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:54.323552 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120038a-c848-4549-9ec4-7e3cb4c36cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:54.323838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120038a-c848-4549-9ec4-7e3cb4c36cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:54.324385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120038a-c848-4549-9ec4-7e3cb4c36cd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:54.325489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120038a-c848-4549-9ec4-7e3cb4c36cd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:54.326003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120038a-c848-4549-9ec4-7e3cb4c36cd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:54.328992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120038a-c848-4549-9ec4-7e3cb4c36cd1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:54.394978 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 868/1745] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:54.398371 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17415e-df47-48f7-84f4-e841b74b558d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:54.398575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17415e-df47-48f7-84f4-e841b74b558d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:54.398852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17415e-df47-48f7-84f4-e841b74b558d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:54.410959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 878/1746] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:54 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:54.412086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6c022-060d-4a98-8a7c-412b068f2320 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:54.425439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9e17415e-df47-48f7-84f4-e841b74b558d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:54.426663 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9e17415e-df47-48f7-84f4-e841b74b558d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:54.430568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6c022-060d-4a98-8a7c-412b068f2320 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:54.435636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 869/1747] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:54.436590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17415e-df47-48f7-84f4-e841b74b558d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:54.439225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9e17415e-df47-48f7-84f4-e841b74b558d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:54.442398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2a704-0d25-4ebb-9677-4a18f1ba161d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:54.443358 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2a704-0d25-4ebb-9677-4a18f1ba161d admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:54.443748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2a704-0d25-4ebb-9677-4a18f1ba161d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:54.444127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2a704-0d25-4ebb-9677-4a18f1ba161d admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:54.444864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2a704-0d25-4ebb-9677-4a18f1ba161d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:54.447615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a2a704-0d25-4ebb-9677-4a18f1ba161d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:54.453691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17415e-df47-48f7-84f4-e841b74b558d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:54.460533 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa43399-caa3-4f99-b6e3-b9dea7c6981d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:54.461294 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa43399-caa3-4f99-b6e3-b9dea7c6981d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:54.461603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa43399-caa3-4f99-b6e3-b9dea7c6981d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:54.461879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa43399-caa3-4f99-b6e3-b9dea7c6981d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:54.462786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa43399-caa3-4f99-b6e3-b9dea7c6981d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:54.464453 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa43399-caa3-4f99-b6e3-b9dea7c6981d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:54.478800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 870/1748] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:54.479748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa43399-caa3-4f99-b6e3-b9dea7c6981d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:54.491072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa43399-caa3-4f99-b6e3-b9dea7c6981d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:54.496956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45eae68-cf26-4139-b797-b5b1fc474a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:54.497616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45eae68-cf26-4139-b797-b5b1fc474a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:54.497953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45eae68-cf26-4139-b797-b5b1fc474a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:54.498225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45eae68-cf26-4139-b797-b5b1fc474a1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:54.498877 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45eae68-cf26-4139-b797-b5b1fc474a1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 09:59:54.499178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45eae68-cf26-4139-b797-b5b1fc474a1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:54.500979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45eae68-cf26-4139-b797-b5b1fc474a1b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:54.509360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:54.516577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 871/1749] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 09:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 09:59:54.534627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45eae68-cf26-4139-b797-b5b1fc474a1b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:54.534627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45eae68-cf26-4139-b797-b5b1fc474a1b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:54.536469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c447af-91e4-4110-b671-0b0bebb5fdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:54.537260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c447af-91e4-4110-b671-0b0bebb5fdc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:54.537549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c447af-91e4-4110-b671-0b0bebb5fdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:54.537830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c447af-91e4-4110-b671-0b0bebb5fdc6 admin admin] PATH_INFO: `/v3/projects/36d2812e93ea4d9d82d2540d3ec4ffa7/users/993624e46def411db786605ff9309f99/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:54.565745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c447af-91e4-4110-b671-0b0bebb5fdc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d2812e93ea4d9d82d2540d3ec4ffa7, user_id=993624e46def411db786605ff9309f99, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:54.567765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c447af-91e4-4110-b671-0b0bebb5fdc6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:54.595622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 879/1750] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:54 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:54.597531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2a704-0d25-4ebb-9677-4a18f1ba161d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:54.650910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 872/1751] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:54 2022] PUT /identity/v3/projects/36d2812e93ea4d9d82d2540d3ec4ffa7/users/993624e46def411db786605ff9309f99/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:54.650910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c447af-91e4-4110-b671-0b0bebb5fdc6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:54.769071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c447af-91e4-4110-b671-0b0bebb5fdc6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:54.774488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedb437-3ce7-44fd-babe-977df315dda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:54.775138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedb437-3ce7-44fd-babe-977df315dda4 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:54.775476 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedb437-3ce7-44fd-babe-977df315dda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:54.775758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedb437-3ce7-44fd-babe-977df315dda4 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:54.776581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedb437-3ce7-44fd-babe-977df315dda4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:54.779502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedb437-3ce7-44fd-babe-977df315dda4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:54.798119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2a704-0d25-4ebb-9677-4a18f1ba161d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:54.819577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe2c0d-d346-4882-91a0-bba8338b0cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:54.820914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe2c0d-d346-4882-91a0-bba8338b0cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:54.820914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe2c0d-d346-4882-91a0-bba8338b0cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:54.820914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe2c0d-d346-4882-91a0-bba8338b0cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:54.821507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe2c0d-d346-4882-91a0-bba8338b0cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:54.823808 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe2c0d-d346-4882-91a0-bba8338b0cfa admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:54.912671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 873/1752] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:54 2022] POST /identity/v3/users => generated 292 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:54.916631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedb437-3ce7-44fd-babe-977df315dda4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:54.937616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 880/1753] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:54 2022] POST /identity/v3/projects => generated 355 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:54.937616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe2c0d-d346-4882-91a0-bba8338b0cfa admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:54.941156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedb437-3ce7-44fd-babe-977df315dda4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:55.007653 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:55.103529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a1475-6270-4324-b363-7a8ea8016366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:55.104189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a1475-6270-4324-b363-7a8ea8016366 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.104473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a1475-6270-4324-b363-7a8ea8016366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.104746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a1475-6270-4324-b363-7a8ea8016366 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.105439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a1475-6270-4324-b363-7a8ea8016366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:55.113002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a1475-6270-4324-b363-7a8ea8016366 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:55.144519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 874/1754] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:55.145802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a1475-6270-4324-b363-7a8ea8016366 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:55.168681 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe2c0d-d346-4882-91a0-bba8338b0cfa admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:55.189573 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95436e-ee21-4394-aacf-9de2e34dae8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:55.190217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95436e-ee21-4394-aacf-9de2e34dae8b admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.190499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95436e-ee21-4394-aacf-9de2e34dae8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.190683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a1475-6270-4324-b363-7a8ea8016366 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:55.190815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95436e-ee21-4394-aacf-9de2e34dae8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.191411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95436e-ee21-4394-aacf-9de2e34dae8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:55.193069 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95436e-ee21-4394-aacf-9de2e34dae8b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:55.209045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e74d1d-c213-44ba-9454-180eceff013e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:55.209704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e74d1d-c213-44ba-9454-180eceff013e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.209967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e74d1d-c213-44ba-9454-180eceff013e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.215728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e74d1d-c213-44ba-9454-180eceff013e admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.215867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 881/1755] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:55.215947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e74d1d-c213-44ba-9454-180eceff013e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:55.216018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95436e-ee21-4394-aacf-9de2e34dae8b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:55.216091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e74d1d-c213-44ba-9454-180eceff013e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:55.258122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95436e-ee21-4394-aacf-9de2e34dae8b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:55.264326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b3846-d5ef-44a5-9a5e-e70e33372d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:55.265270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b3846-d5ef-44a5-9a5e-e70e33372d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.265764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b3846-d5ef-44a5-9a5e-e70e33372d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.266179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b3846-d5ef-44a5-9a5e-e70e33372d4d admin admin] PATH_INFO: `/v3/projects/36d2812e93ea4d9d82d2540d3ec4ffa7/users/993624e46def411db786605ff9309f99/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.270558 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b3846-d5ef-44a5-9a5e-e70e33372d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d2812e93ea4d9d82d2540d3ec4ffa7, user_id=993624e46def411db786605ff9309f99, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:55.274224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b3846-d5ef-44a5-9a5e-e70e33372d4d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:55.290063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 882/1756] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:55 2022] PUT /identity/v3/projects/36d2812e93ea4d9d82d2540d3ec4ffa7/users/993624e46def411db786605ff9309f99/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:55.294816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b3846-d5ef-44a5-9a5e-e70e33372d4d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:55.348420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 875/1757] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:55 2022] POST /identity/v3/users => generated 278 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:55.350665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5dc7-a120-4380-baa4-36e5be6a4671 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.350870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5dc7-a120-4380-baa4-36e5be6a4671 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.351114 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5dc7-a120-4380-baa4-36e5be6a4671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.439568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2b3c5dc7-a120-4380-baa4-36e5be6a4671 None None] MFA Rules not processed for user `993624e46def411db786605ff9309f99`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:55.440933 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2b3c5dc7-a120-4380-baa4-36e5be6a4671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:55.458063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b3846-d5ef-44a5-9a5e-e70e33372d4d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:55.466438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ee714-b4f9-4a02-8e1e-c0d5f00fe68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:55.467733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ee714-b4f9-4a02-8e1e-c0d5f00fe68c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.468018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ee714-b4f9-4a02-8e1e-c0d5f00fe68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.468296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ee714-b4f9-4a02-8e1e-c0d5f00fe68c admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.469195 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ee714-b4f9-4a02-8e1e-c0d5f00fe68c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:55.471179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ee714-b4f9-4a02-8e1e-c0d5f00fe68c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:55.487024 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 883/1758] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:55.488032 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ee714-b4f9-4a02-8e1e-c0d5f00fe68c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:55.499743 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ee714-b4f9-4a02-8e1e-c0d5f00fe68c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:55.513882 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905ba0b-562e-4576-b327-bcd7d1a350ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:55.515069 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905ba0b-562e-4576-b327-bcd7d1a350ab admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.515545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905ba0b-562e-4576-b327-bcd7d1a350ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.516294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905ba0b-562e-4576-b327-bcd7d1a350ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.517113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905ba0b-562e-4576-b327-bcd7d1a350ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:55.518908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905ba0b-562e-4576-b327-bcd7d1a350ab admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:55.533814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 884/1759] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:55.535163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905ba0b-562e-4576-b327-bcd7d1a350ab admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:55.549187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905ba0b-562e-4576-b327-bcd7d1a350ab admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:55.555741 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 876/1760] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:55 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:55.556597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c5dc7-a120-4380-baa4-36e5be6a4671 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:55.563192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860535f-ce0c-4a41-a5d3-8e045ee48099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:55.565003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860535f-ce0c-4a41-a5d3-8e045ee48099 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.565794 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860535f-ce0c-4a41-a5d3-8e045ee48099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.566227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860535f-ce0c-4a41-a5d3-8e045ee48099 admin admin] PATH_INFO: `/v3/projects/4c7dcfeb86c045bcace8978462b1410f/users/f5fb037fbc3d4b94927613366a58e596/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.569125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c5dc7-a120-4380-baa4-36e5be6a4671 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:55.575221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c45326-5114-4b02-8e46-2e0262479d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:55.575863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c45326-5114-4b02-8e46-2e0262479d10 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.576151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c45326-5114-4b02-8e46-2e0262479d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.576423 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c45326-5114-4b02-8e46-2e0262479d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.577224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c45326-5114-4b02-8e46-2e0262479d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:55.579587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c45326-5114-4b02-8e46-2e0262479d10 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:55.592297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 877/1761] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:55.596930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c45326-5114-4b02-8e46-2e0262479d10 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:55.597710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860535f-ce0c-4a41-a5d3-8e045ee48099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7dcfeb86c045bcace8978462b1410f, user_id=f5fb037fbc3d4b94927613366a58e596, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:55.598978 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-26c45326-5114-4b02-8e46-2e0262479d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:55.600448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860535f-ce0c-4a41-a5d3-8e045ee48099 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:55.676355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c45326-5114-4b02-8e46-2e0262479d10 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:55.680427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bac9b1-fac3-4de8-9a0c-26cb27c1390e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:55.681122 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bac9b1-fac3-4de8-9a0c-26cb27c1390e service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.681366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bac9b1-fac3-4de8-9a0c-26cb27c1390e service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.681639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bac9b1-fac3-4de8-9a0c-26cb27c1390e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.692795 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bac9b1-fac3-4de8-9a0c-26cb27c1390e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:55.694301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bac9b1-fac3-4de8-9a0c-26cb27c1390e service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:55.707335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 878/1762] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:55.708192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bac9b1-fac3-4de8-9a0c-26cb27c1390e service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:55.720999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bac9b1-fac3-4de8-9a0c-26cb27c1390e service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:55.726626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 885/1763] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:55 2022] PUT /identity/v3/projects/4c7dcfeb86c045bcace8978462b1410f/users/f5fb037fbc3d4b94927613366a58e596/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:55.736585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d7358-6521-44f4-8cd8-f02bd4352c56 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.736937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d7358-6521-44f4-8cd8-f02bd4352c56 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.737312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d7358-6521-44f4-8cd8-f02bd4352c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.767473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ea1d7358-6521-44f4-8cd8-f02bd4352c56 None None] MFA Rules not processed for user `f5fb037fbc3d4b94927613366a58e596`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:55.769213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ea1d7358-6521-44f4-8cd8-f02bd4352c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:55.796286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871dbfce-0d24-4f25-9d87-f6ea4dc270fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:55.797004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dbfce-0d24-4f25-9d87-f6ea4dc270fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.797695 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dbfce-0d24-4f25-9d87-f6ea4dc270fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.797992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871dbfce-0d24-4f25-9d87-f6ea4dc270fc admin admin] PATH_INFO: `/v3/projects/b8d6cd1ee98d49b2a31cd74eec99ac7b/users/24fc56655181409ca2ecd89adc824450/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:55.816407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871dbfce-0d24-4f25-9d87-f6ea4dc270fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d6cd1ee98d49b2a31cd74eec99ac7b, user_id=24fc56655181409ca2ecd89adc824450, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:55.819799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871dbfce-0d24-4f25-9d87-f6ea4dc270fc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:55.853413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 886/1764] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 09:59:55.866503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d7358-6521-44f4-8cd8-f02bd4352c56 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:55.948856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d7358-6521-44f4-8cd8-f02bd4352c56 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:55.952861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 879/1765] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:55 2022] PUT /identity/v3/projects/b8d6cd1ee98d49b2a31cd74eec99ac7b/users/24fc56655181409ca2ecd89adc824450/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:55.955947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cb32e-59c5-4a6c-94ae-b74de8003077 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:55.956292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cb32e-59c5-4a6c-94ae-b74de8003077 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:55.956580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cb32e-59c5-4a6c-94ae-b74de8003077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:56.000580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c74cb32e-59c5-4a6c-94ae-b74de8003077 None None] MFA Rules not processed for user `24fc56655181409ca2ecd89adc824450`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:56.002258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c74cb32e-59c5-4a6c-94ae-b74de8003077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:56.013582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:56.054015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa7313-c735-4584-b2e8-a61dd5a6b115 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:56.057388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa7313-c735-4584-b2e8-a61dd5a6b115 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:56.057673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa7313-c735-4584-b2e8-a61dd5a6b115 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:56.060108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa7313-c735-4584-b2e8-a61dd5a6b115 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:56.146466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 880/1766] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:56.147318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cb32e-59c5-4a6c-94ae-b74de8003077 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:56.149841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c74cb32e-59c5-4a6c-94ae-b74de8003077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:56.162493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa7313-c735-4584-b2e8-a61dd5a6b115 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:56.164049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cb32e-59c5-4a6c-94ae-b74de8003077 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:56.166012 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa7313-c735-4584-b2e8-a61dd5a6b115 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:56.167644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15218ff-7e1a-4371-8e1d-5263aaa0f673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:56.168351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15218ff-7e1a-4371-8e1d-5263aaa0f673 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:56.168698 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15218ff-7e1a-4371-8e1d-5263aaa0f673 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:56.169206 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15218ff-7e1a-4371-8e1d-5263aaa0f673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:56.171082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b15218ff-7e1a-4371-8e1d-5263aaa0f673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:56.180650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 887/1767] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 09:59:56.186643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa7313-c735-4584-b2e8-a61dd5a6b115 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:56.273687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15218ff-7e1a-4371-8e1d-5263aaa0f673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:56.275613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15218ff-7e1a-4371-8e1d-5263aaa0f673 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:56.324042 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 881/1768] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:56.347280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa7313-c735-4584-b2e8-a61dd5a6b115 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:56.360337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b96e9-bd7a-4f21-b59c-1092de765df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:56.366015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b96e9-bd7a-4f21-b59c-1092de765df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:56.366015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b96e9-bd7a-4f21-b59c-1092de765df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:56.366015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:56.366015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b96e9-bd7a-4f21-b59c-1092de765df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:56.366015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b96e9-bd7a-4f21-b59c-1092de765df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:56.367934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b96e9-bd7a-4f21-b59c-1092de765df1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:56.484705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 888/1769] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 09:59:56 2022] POST /identity/v3/projects => generated 355 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:56.489322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15218ff-7e1a-4371-8e1d-5263aaa0f673 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:56.505689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15218ff-7e1a-4371-8e1d-5263aaa0f673 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:56.689612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7224d3-1658-40fe-9711-28e90d17b555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:56.690774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7224d3-1658-40fe-9711-28e90d17b555 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:56.691130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7224d3-1658-40fe-9711-28e90d17b555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:56.691505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7224d3-1658-40fe-9711-28e90d17b555 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:56.692386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7224d3-1658-40fe-9711-28e90d17b555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:56.695304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7224d3-1658-40fe-9711-28e90d17b555 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:56.838309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 882/1770] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 09:59:56 2022] POST /identity/v3/users => generated 284 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 09:59:56.841511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b96e9-bd7a-4f21-b59c-1092de765df1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:56.853641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b96e9-bd7a-4f21-b59c-1092de765df1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:56.859163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed325f7c-643e-4473-aa3f-80b4d55165a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:56.859776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed325f7c-643e-4473-aa3f-80b4d55165a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:56.860084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed325f7c-643e-4473-aa3f-80b4d55165a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:56.860368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed325f7c-643e-4473-aa3f-80b4d55165a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:56.861174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed325f7c-643e-4473-aa3f-80b4d55165a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:56.863207 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed325f7c-643e-4473-aa3f-80b4d55165a1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:56.870019 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27db13-24da-4ab4-90df-be1d065b2457 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:56.870587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27db13-24da-4ab4-90df-be1d065b2457 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:56.871088 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27db13-24da-4ab4-90df-be1d065b2457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:56.879067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 889/1771] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:56.888733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed325f7c-643e-4473-aa3f-80b4d55165a1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:56.900591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed325f7c-643e-4473-aa3f-80b4d55165a1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:56.907855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e707d-8fa2-43c3-8130-7234a390ef8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:56.908859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e707d-8fa2-43c3-8130-7234a390ef8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:56.909271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e707d-8fa2-43c3-8130-7234a390ef8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:56.909642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e707d-8fa2-43c3-8130-7234a390ef8e admin admin] PATH_INFO: `/v3/projects/482fd8bc50e74dd49c8f4231de1658cf/users/264cf761ea7b4c9a9921edce8aeaceea/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:56.922499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:56.927480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e707d-8fa2-43c3-8130-7234a390ef8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=482fd8bc50e74dd49c8f4231de1658cf, user_id=264cf761ea7b4c9a9921edce8aeaceea, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:56.930869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e707d-8fa2-43c3-8130-7234a390ef8e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:56.984949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-aa27db13-24da-4ab4-90df-be1d065b2457 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:56.986077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-aa27db13-24da-4ab4-90df-be1d065b2457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:57.050543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 890/1772] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:56 2022] PUT /identity/v3/projects/482fd8bc50e74dd49c8f4231de1658cf/users/264cf761ea7b4c9a9921edce8aeaceea/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:57.054145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e707d-8fa2-43c3-8130-7234a390ef8e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:57.102947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 883/1773] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 09:59:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:57.213460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e707d-8fa2-43c3-8130-7234a390ef8e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:57.219130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360941d1-02a1-4fd1-bac4-94491dd55a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:57.220457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360941d1-02a1-4fd1-bac4-94491dd55a0d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:57.220818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360941d1-02a1-4fd1-bac4-94491dd55a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:57.221197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360941d1-02a1-4fd1-bac4-94491dd55a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:57.221928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360941d1-02a1-4fd1-bac4-94491dd55a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:57.223757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360941d1-02a1-4fd1-bac4-94491dd55a0d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:57.240956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 891/1774] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 09:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 09:59:57.244625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27db13-24da-4ab4-90df-be1d065b2457 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:57.273126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27db13-24da-4ab4-90df-be1d065b2457 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:57.280503 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880959cc-f8dc-4518-8433-d20bceca22fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:57.282031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880959cc-f8dc-4518-8433-d20bceca22fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:57.282475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880959cc-f8dc-4518-8433-d20bceca22fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:57.282892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880959cc-f8dc-4518-8433-d20bceca22fe admin admin] PATH_INFO: `/v3/projects/482fd8bc50e74dd49c8f4231de1658cf/users/264cf761ea7b4c9a9921edce8aeaceea/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:57.290122 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880959cc-f8dc-4518-8433-d20bceca22fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=482fd8bc50e74dd49c8f4231de1658cf, user_id=264cf761ea7b4c9a9921edce8aeaceea, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:57.294629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880959cc-f8dc-4518-8433-d20bceca22fe admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:57.332566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 884/1775] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 09:59:57 2022] PUT /identity/v3/projects/482fd8bc50e74dd49c8f4231de1658cf/users/264cf761ea7b4c9a9921edce8aeaceea/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 09:59:57.338177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:57.342879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65c6e1-3951-44db-91ee-a962f929332b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:57.343439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65c6e1-3951-44db-91ee-a962f929332b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:57.344131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65c6e1-3951-44db-91ee-a962f929332b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:57.421944 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-be65c6e1-3951-44db-91ee-a962f929332b None None] MFA Rules not processed for user `264cf761ea7b4c9a9921edce8aeaceea`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:57.424005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-be65c6e1-3951-44db-91ee-a962f929332b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:57.544836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 892/1776] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 09:59:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:57.553004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880959cc-f8dc-4518-8433-d20bceca22fe admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:57.553165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65c6e1-3951-44db-91ee-a962f929332b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:57.555385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-880959cc-f8dc-4518-8433-d20bceca22fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:57.672626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65c6e1-3951-44db-91ee-a962f929332b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:57.691553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930f616-23c8-4fd3-81ad-d113e21dbfed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:57.692233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930f616-23c8-4fd3-81ad-d113e21dbfed service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:57.695923 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930f616-23c8-4fd3-81ad-d113e21dbfed service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:57.695923 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930f616-23c8-4fd3-81ad-d113e21dbfed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:57.724521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930f616-23c8-4fd3-81ad-d113e21dbfed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:57.728980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930f616-23c8-4fd3-81ad-d113e21dbfed service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:57.768150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 893/1777] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:57.794779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880959cc-f8dc-4518-8433-d20bceca22fe admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:57.808950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac6d08-72fc-4883-b322-2c1e3ab3d34a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:57.809872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac6d08-72fc-4883-b322-2c1e3ab3d34a service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:57.810569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac6d08-72fc-4883-b322-2c1e3ab3d34a service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:57.810845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac6d08-72fc-4883-b322-2c1e3ab3d34a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:57.972159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac6d08-72fc-4883-b322-2c1e3ab3d34a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:57.973808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac6d08-72fc-4883-b322-2c1e3ab3d34a service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:57.997828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 885/1778] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 09:59:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 09:59:58.076848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:58.355003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930f616-23c8-4fd3-81ad-d113e21dbfed service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:58.372149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930f616-23c8-4fd3-81ad-d113e21dbfed service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:58.388359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac03ae7-604a-4a26-a8b1-7591ca83ac46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:58.389286 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac03ae7-604a-4a26-a8b1-7591ca83ac46 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:58.389569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac03ae7-604a-4a26-a8b1-7591ca83ac46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:58.389883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac03ae7-604a-4a26-a8b1-7591ca83ac46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:58.524136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac03ae7-604a-4a26-a8b1-7591ca83ac46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:58.528911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac03ae7-604a-4a26-a8b1-7591ca83ac46 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:58.575875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 894/1779] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:58.683141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77987ee2-89be-4f30-8090-5713551fba91 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:58.683658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77987ee2-89be-4f30-8090-5713551fba91 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:58.684107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77987ee2-89be-4f30-8090-5713551fba91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:58.737808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-77987ee2-89be-4f30-8090-5713551fba91 None None] MFA Rules not processed for user `4fc6c3d1ce8645808c22d1b59b15a100`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 09:59:58.746968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-77987ee2-89be-4f30-8090-5713551fba91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:58.818744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:59.009384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 886/1780] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Aug 11 09:59:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 09:59:59.269030 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac03ae7-604a-4a26-a8b1-7591ca83ac46 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 09:59:59.270586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8ac03ae7-604a-4a26-a8b1-7591ca83ac46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 09:59:59.286633 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac03ae7-604a-4a26-a8b1-7591ca83ac46 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 09:59:59.304562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d6f85-8329-48f7-8f11-bf36eb9f4eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 09:59:59.305372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6f85-8329-48f7-8f11-bf36eb9f4eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 09:59:59.305728 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6f85-8329-48f7-8f11-bf36eb9f4eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 09:59:59.306064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d6f85-8329-48f7-8f11-bf36eb9f4eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 09:59:59.316126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d6f85-8329-48f7-8f11-bf36eb9f4eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 09:59:59.317958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d6f85-8329-48f7-8f11-bf36eb9f4eb5 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 09:59:59.347197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 895/1781] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 09:59:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 09:59:59.385165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 09:59:59.504786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:01.965007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdac46-a5c6-41d5-b979-28d5f22296f1 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:01.966945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdac46-a5c6-41d5-b979-28d5f22296f1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:01.966945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdac46-a5c6-41d5-b979-28d5f22296f1 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:01.967797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 887/1782] 10.222.0.17 () {54 vars in 921 bytes} [Thu Aug 11 10:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:00:01.974315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d090-fce1-4f47-8b85-c2646760dbbc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:01.974866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d090-fce1-4f47-8b85-c2646760dbbc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:01.975209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144d090-fce1-4f47-8b85-c2646760dbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:02.041838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0144d090-fce1-4f47-8b85-c2646760dbbc None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:02.041838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0144d090-fce1-4f47-8b85-c2646760dbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:02.177649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 896/1783] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Aug 11 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:00:02.189152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdac46-a5c6-41d5-b979-28d5f22296f1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:02.195201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d5fdac46-a5c6-41d5-b979-28d5f22296f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:02.233477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fdac46-a5c6-41d5-b979-28d5f22296f1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:02.242755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25c59e-b0bf-4bed-9eef-ba42b07b7999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:02.243387 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25c59e-b0bf-4bed-9eef-ba42b07b7999 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:02.243669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25c59e-b0bf-4bed-9eef-ba42b07b7999 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:02.243942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25c59e-b0bf-4bed-9eef-ba42b07b7999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:02.253740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25c59e-b0bf-4bed-9eef-ba42b07b7999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:02.257139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25c59e-b0bf-4bed-9eef-ba42b07b7999 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:02.266820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 888/1784] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:02.400514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c982aba-21bc-4beb-ab50-d0a6eb7d5309 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:02.400730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c982aba-21bc-4beb-ab50-d0a6eb7d5309 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:02.401002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c982aba-21bc-4beb-ab50-d0a6eb7d5309 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:02.402448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 897/1785] 10.222.0.17 () {54 vars in 921 bytes} [Thu Aug 11 10:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:00:02.413033 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be240d-047d-4213-a0b4-227e39b00a9e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:02.413310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be240d-047d-4213-a0b4-227e39b00a9e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:02.413589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be240d-047d-4213-a0b4-227e39b00a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:02.434073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-99be240d-047d-4213-a0b4-227e39b00a9e None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:02.435299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-99be240d-047d-4213-a0b4-227e39b00a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:02.445697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 889/1786] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Aug 11 10:00:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:02.458035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c982aba-21bc-4beb-ab50-d0a6eb7d5309 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:02.462106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0c982aba-21bc-4beb-ab50-d0a6eb7d5309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:02.477943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c982aba-21bc-4beb-ab50-d0a6eb7d5309 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:02.483446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7da671-1a08-400a-8a53-c0b54e081792 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:02.484780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7da671-1a08-400a-8a53-c0b54e081792 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:02.485066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7da671-1a08-400a-8a53-c0b54e081792 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:02.485395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7da671-1a08-400a-8a53-c0b54e081792 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:02.494684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7da671-1a08-400a-8a53-c0b54e081792 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:02.496246 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7da671-1a08-400a-8a53-c0b54e081792 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:02.502587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:02.507864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 898/1787] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:00:09.022791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be240d-047d-4213-a0b4-227e39b00a9e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:09.184210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be240d-047d-4213-a0b4-227e39b00a9e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:09.191629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:09.199107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:09.199936 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:09.200441 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] PATH_INFO: `/v3/users/f5fb037fbc3d4b94927613366a58e596` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:09.202105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5fb037fbc3d4b94927613366a58e596)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:09.218998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:09.958131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5fb037fbc3d4b94927613366a58e596'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:09.958826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5fb037fbc3d4b94927613366a58e596'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:09.959434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5fb037fbc3d4b94927613366a58e596'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:10.057679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5fb037fbc3d4b94927613366a58e596'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:10.070383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5fb037fbc3d4b94927613366a58e596'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:10.083462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 890/1788] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:09 2022] DELETE /identity/v3/users/f5fb037fbc3d4b94927613366a58e596 => generated 0 bytes in 1061 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:10.089700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea733e-d3fd-49f5-b1ac-96a1772f0c92 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:10.090120 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea733e-d3fd-49f5-b1ac-96a1772f0c92 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:10.090120 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea733e-d3fd-49f5-b1ac-96a1772f0c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:10.113216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7cea733e-d3fd-49f5-b1ac-96a1772f0c92 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:10.116805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7cea733e-d3fd-49f5-b1ac-96a1772f0c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:10.204359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 899/1789] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:10.215074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:10.216546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:10.279539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:10.284402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec5937-4bfc-4081-8808-128bac17392e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:10.288636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75734d-6877-4738-9c49-0fc7b931ab40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:10.289362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75734d-6877-4738-9c49-0fc7b931ab40 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:10.289578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75734d-6877-4738-9c49-0fc7b931ab40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:10.293329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75734d-6877-4738-9c49-0fc7b931ab40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:10.300595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75734d-6877-4738-9c49-0fc7b931ab40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:10.302254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75734d-6877-4738-9c49-0fc7b931ab40 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:10.326807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 891/1790] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:11.738153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea733e-d3fd-49f5-b1ac-96a1772f0c92 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:11.739996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7cea733e-d3fd-49f5-b1ac-96a1772f0c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:11.755465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea733e-d3fd-49f5-b1ac-96a1772f0c92 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:11.761445 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:11.764298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:11.764298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:11.764298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] PATH_INFO: `/v3/projects/4c7dcfeb86c045bcace8978462b1410f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:11.764801 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c7dcfeb86c045bcace8978462b1410f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:11.767986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:11.898922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c7dcfeb86c045bcace8978462b1410f', 'request_id': 'req-5c65bded-dbe5-4e0e-86ea-1de864e65e09'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:11.909424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c7dcfeb86c045bcace8978462b1410f', 'request_id': 'req-5c65bded-dbe5-4e0e-86ea-1de864e65e09'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:11.917800 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c7dcfeb86c045bcace8978462b1410f', 'request_id': 'req-5c65bded-dbe5-4e0e-86ea-1de864e65e09'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:12.034412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] The token cache is being invalidate because project 4c7dcfeb86c045bcace8978462b1410f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:12.034412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:12.034412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5c65bded-dbe5-4e0e-86ea-1de864e65e09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:12.034412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 900/1791] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:11 2022] DELETE /identity/v3/projects/4c7dcfeb86c045bcace8978462b1410f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:12.039036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4486104-fb00-4f28-a3ce-22b08de9457e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:12.039499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4486104-fb00-4f28-a3ce-22b08de9457e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:12.040281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4486104-fb00-4f28-a3ce-22b08de9457e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:12.073497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c4486104-fb00-4f28-a3ce-22b08de9457e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:12.073497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c4486104-fb00-4f28-a3ce-22b08de9457e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:12.162460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 892/1792] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:00:12.165558 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699d8aa-da2a-4add-a682-80d14ca61550 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:12.165924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699d8aa-da2a-4add-a682-80d14ca61550 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:12.166272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699d8aa-da2a-4add-a682-80d14ca61550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:12.198095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c699d8aa-da2a-4add-a682-80d14ca61550 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:12.199304 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c699d8aa-da2a-4add-a682-80d14ca61550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:12.213059 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 901/1793] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:12.222694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4486104-fb00-4f28-a3ce-22b08de9457e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:12.235205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4486104-fb00-4f28-a3ce-22b08de9457e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:12.240746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6367a6-aef1-4526-a2fb-f12f51d32bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:12.243476 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6367a6-aef1-4526-a2fb-f12f51d32bba admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:12.243476 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6367a6-aef1-4526-a2fb-f12f51d32bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:12.243476 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6367a6-aef1-4526-a2fb-f12f51d32bba admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:12.243476 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6367a6-aef1-4526-a2fb-f12f51d32bba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:00:12.243476 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6367a6-aef1-4526-a2fb-f12f51d32bba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:12.245145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6367a6-aef1-4526-a2fb-f12f51d32bba admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:12.258505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 893/1794] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:00:12.263207 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699d8aa-da2a-4add-a682-80d14ca61550 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:12.277826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699d8aa-da2a-4add-a682-80d14ca61550 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:12.285524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9572f81-7125-4bff-b54a-8d56c80955e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:12.286388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9572f81-7125-4bff-b54a-8d56c80955e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:12.286809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9572f81-7125-4bff-b54a-8d56c80955e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:12.287239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9572f81-7125-4bff-b54a-8d56c80955e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:12.288013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9572f81-7125-4bff-b54a-8d56c80955e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:12.290482 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9572f81-7125-4bff-b54a-8d56c80955e9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:12.426848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 902/1795] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:12 2022] POST /identity/v3/projects => generated 369 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:12.428156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6367a6-aef1-4526-a2fb-f12f51d32bba admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:12.439436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6367a6-aef1-4526-a2fb-f12f51d32bba admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:12.521781 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840751f-e201-4465-af67-3a0817f0acf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:12.524575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840751f-e201-4465-af67-3a0817f0acf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:12.524870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840751f-e201-4465-af67-3a0817f0acf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:12.526164 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0840751f-e201-4465-af67-3a0817f0acf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:12.526831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840751f-e201-4465-af67-3a0817f0acf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:12.528789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0840751f-e201-4465-af67-3a0817f0acf6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:12.684060 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 894/1796] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:12 2022] POST /identity/v3/users => generated 285 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:12.688450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9572f81-7125-4bff-b54a-8d56c80955e9 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:12.708076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9572f81-7125-4bff-b54a-8d56c80955e9 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:12.713524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f03073-ec2e-4c3e-8a70-6d7bdb3734ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:12.714396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f03073-ec2e-4c3e-8a70-6d7bdb3734ed admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:12.714884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f03073-ec2e-4c3e-8a70-6d7bdb3734ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:12.715486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f03073-ec2e-4c3e-8a70-6d7bdb3734ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:12.716378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f03073-ec2e-4c3e-8a70-6d7bdb3734ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:12.719462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f03073-ec2e-4c3e-8a70-6d7bdb3734ed admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:12.743768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 903/1797] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:12.747742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840751f-e201-4465-af67-3a0817f0acf6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:12.759336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0840751f-e201-4465-af67-3a0817f0acf6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:12.764208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0459254-d99a-413f-a1bd-64c58bd89403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:12.765639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0459254-d99a-413f-a1bd-64c58bd89403 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:12.766096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0459254-d99a-413f-a1bd-64c58bd89403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:12.766382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0459254-d99a-413f-a1bd-64c58bd89403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:12.767034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0459254-d99a-413f-a1bd-64c58bd89403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:12.768639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0459254-d99a-413f-a1bd-64c58bd89403 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:12.779974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 895/1798] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:12.784744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f03073-ec2e-4c3e-8a70-6d7bdb3734ed admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:12.795434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f03073-ec2e-4c3e-8a70-6d7bdb3734ed admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:12.801442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096c1ff-5e8c-48ca-a057-4c315ded677c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:12.802128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096c1ff-5e8c-48ca-a057-4c315ded677c admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:12.802410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096c1ff-5e8c-48ca-a057-4c315ded677c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:12.802686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096c1ff-5e8c-48ca-a057-4c315ded677c admin admin] PATH_INFO: `/v3/projects/78ec9e6bf0644cb7a8e3833b3f28fc4e/users/2ed9126ca8274ed291fd6c36b374576b/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:12.823420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096c1ff-5e8c-48ca-a057-4c315ded677c admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ec9e6bf0644cb7a8e3833b3f28fc4e, user_id=2ed9126ca8274ed291fd6c36b374576b, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:12.826158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096c1ff-5e8c-48ca-a057-4c315ded677c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:12.970444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 904/1799] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:12 2022] PUT /identity/v3/projects/78ec9e6bf0644cb7a8e3833b3f28fc4e/users/2ed9126ca8274ed291fd6c36b374576b/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:12.976838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfb343-36a5-43b3-a403-32407d390263 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:12.977309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfb343-36a5-43b3-a403-32407d390263 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:12.977885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfb343-36a5-43b3-a403-32407d390263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:13.015443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9adfb343-36a5-43b3-a403-32407d390263 None None] MFA Rules not processed for user `2ed9126ca8274ed291fd6c36b374576b`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:13.016758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9adfb343-36a5-43b3-a403-32407d390263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:13.067517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:13.094657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 896/1800] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:13.107060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096c1ff-5e8c-48ca-a057-4c315ded677c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:13.110956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2096c1ff-5e8c-48ca-a057-4c315ded677c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:13.157217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:13.222068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096c1ff-5e8c-48ca-a057-4c315ded677c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:13.226042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c0966-33cd-4a08-b6f9-1853c8ce4dd9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83448e4c6704f7c8771132bd67eaa1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:13.226911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c0966-33cd-4a08-b6f9-1853c8ce4dd9 service glance] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:13.227194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c0966-33cd-4a08-b6f9-1853c8ce4dd9 service glance] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:13.228162 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c0966-33cd-4a08-b6f9-1853c8ce4dd9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:13.237312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c0966-33cd-4a08-b6f9-1853c8ce4dd9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:13.238838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c0966-33cd-4a08-b6f9-1853c8ce4dd9 service glance] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:13.253270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 905/1801] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:13.963515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfb343-36a5-43b3-a403-32407d390263 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:13.965549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9adfb343-36a5-43b3-a403-32407d390263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:14.043963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfb343-36a5-43b3-a403-32407d390263 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:14.056571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:14.056571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:14.056571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:14.056571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] PATH_INFO: `/v3/users/1688e9a113a746889abf85084d4ec309` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:14.057050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1688e9a113a746889abf85084d4ec309)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:14.057050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:14.296757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1688e9a113a746889abf85084d4ec309'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:14.298384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1688e9a113a746889abf85084d4ec309'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:14.299238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1688e9a113a746889abf85084d4ec309'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:14.395106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1688e9a113a746889abf85084d4ec309'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:14.409111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1688e9a113a746889abf85084d4ec309'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:14.423528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 897/1802] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:13 2022] DELETE /identity/v3/users/1688e9a113a746889abf85084d4ec309 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:14.427771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b741565-3aec-48c0-9ecc-e4c511823502 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:14.428241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b741565-3aec-48c0-9ecc-e4c511823502 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:14.428331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b741565-3aec-48c0-9ecc-e4c511823502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:14.449459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5b741565-3aec-48c0-9ecc-e4c511823502 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:14.449459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5b741565-3aec-48c0-9ecc-e4c511823502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:14.524833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 906/1803] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:14.536979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:14.538841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:14.615203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08d6ff-e5d1-469a-99fa-c76cd1943199 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:14.622441 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c954b5-4a90-4f32-887b-8c81fab08985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:14.622441 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c954b5-4a90-4f32-887b-8c81fab08985 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:14.622441 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c954b5-4a90-4f32-887b-8c81fab08985 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:14.622441 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c954b5-4a90-4f32-887b-8c81fab08985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:14.633328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c954b5-4a90-4f32-887b-8c81fab08985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:14.635190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c954b5-4a90-4f32-887b-8c81fab08985 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:14.653966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 898/1804] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:15.259626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b741565-3aec-48c0-9ecc-e4c511823502 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:15.261300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5b741565-3aec-48c0-9ecc-e4c511823502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:15.291401 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b741565-3aec-48c0-9ecc-e4c511823502 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:15.299261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:15.299906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:15.300220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:15.300453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] PATH_INFO: `/v3/projects/66d476cb3df149d48a0186e7aaf85808` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:15.301659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] RBAC: Authorizing `identity:delete_project(project_id=66d476cb3df149d48a0186e7aaf85808)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:15.303663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:15.449296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66d476cb3df149d48a0186e7aaf85808', 'request_id': 'req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:15.450543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66d476cb3df149d48a0186e7aaf85808', 'request_id': 'req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:15.458511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66d476cb3df149d48a0186e7aaf85808', 'request_id': 'req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:15.623126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] The token cache is being invalidate because project 66d476cb3df149d48a0186e7aaf85808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:15.623421 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:15.624212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e45d5c88-de1e-47e3-b8a7-33a47fe0c20b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:15.631681 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 907/1805] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:15 2022] DELETE /identity/v3/projects/66d476cb3df149d48a0186e7aaf85808 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:15.634302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7211e6-4a52-4e57-a29b-a49342387137 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:15.634826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7211e6-4a52-4e57-a29b-a49342387137 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:15.634826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7211e6-4a52-4e57-a29b-a49342387137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:15.669887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3c7211e6-4a52-4e57-a29b-a49342387137 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:15.673669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3c7211e6-4a52-4e57-a29b-a49342387137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:15.678618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:15.789303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 899/1806] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:15.798113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084fb61-8829-4be3-bf22-169de2928fe3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:15.798405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084fb61-8829-4be3-bf22-169de2928fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:15.798683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084fb61-8829-4be3-bf22-169de2928fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:15.861895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2084fb61-8829-4be3-bf22-169de2928fe3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:15.865219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2084fb61-8829-4be3-bf22-169de2928fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:15.877616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 908/1807] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:15.888805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7211e6-4a52-4e57-a29b-a49342387137 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:15.907649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7211e6-4a52-4e57-a29b-a49342387137 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:15.915074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f001d-2b74-4dd4-86c4-ebccdffe89a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:15.915801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f001d-2b74-4dd4-86c4-ebccdffe89a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:15.916343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f001d-2b74-4dd4-86c4-ebccdffe89a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:15.916631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f001d-2b74-4dd4-86c4-ebccdffe89a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:15.917951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f001d-2b74-4dd4-86c4-ebccdffe89a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:00:15.918320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f001d-2b74-4dd4-86c4-ebccdffe89a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:15.923332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f001d-2b74-4dd4-86c4-ebccdffe89a6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:15.945052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 900/1808] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:00:15.950259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084fb61-8829-4be3-bf22-169de2928fe3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:15.960300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084fb61-8829-4be3-bf22-169de2928fe3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:15.980800 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac4c40-8ffd-44f3-b47a-638ae3c73109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:15.981378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac4c40-8ffd-44f3-b47a-638ae3c73109 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:15.981722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac4c40-8ffd-44f3-b47a-638ae3c73109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:15.989381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac4c40-8ffd-44f3-b47a-638ae3c73109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:15.989381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac4c40-8ffd-44f3-b47a-638ae3c73109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:15.989381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac4c40-8ffd-44f3-b47a-638ae3c73109 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:16.097992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:16.121440 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 909/1809] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:15 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:16.122868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f001d-2b74-4dd4-86c4-ebccdffe89a6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:16.144653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f001d-2b74-4dd4-86c4-ebccdffe89a6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:16.245071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c581444-a1a9-48a1-9f70-e3f795678de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:16.247468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c581444-a1a9-48a1-9f70-e3f795678de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:16.247468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c581444-a1a9-48a1-9f70-e3f795678de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:16.247468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c581444-a1a9-48a1-9f70-e3f795678de8 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:16.247468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c581444-a1a9-48a1-9f70-e3f795678de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:16.250211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c581444-a1a9-48a1-9f70-e3f795678de8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:16.364159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac4c40-8ffd-44f3-b47a-638ae3c73109 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:16.374177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 901/1810] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:16 2022] POST /identity/v3/users => generated 294 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:00:16.379107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac4c40-8ffd-44f3-b47a-638ae3c73109 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:16.391526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c208a-216b-477b-b97b-a05a02a68676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:16.391526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c208a-216b-477b-b97b-a05a02a68676 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:16.391526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c208a-216b-477b-b97b-a05a02a68676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:16.391526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c208a-216b-477b-b97b-a05a02a68676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:16.391942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c208a-216b-477b-b97b-a05a02a68676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:16.391942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c208a-216b-477b-b97b-a05a02a68676 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:16.404337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 910/1811] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:16.409405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c581444-a1a9-48a1-9f70-e3f795678de8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:16.424081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c581444-a1a9-48a1-9f70-e3f795678de8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:16.440999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c94a0-d023-4925-9c71-48e3be67dab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:16.442011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c94a0-d023-4925-9c71-48e3be67dab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:16.442652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c94a0-d023-4925-9c71-48e3be67dab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:16.443128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c94a0-d023-4925-9c71-48e3be67dab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:16.444025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c94a0-d023-4925-9c71-48e3be67dab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:16.446116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c94a0-d023-4925-9c71-48e3be67dab1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:16.483321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 902/1812] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:16.491765 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c208a-216b-477b-b97b-a05a02a68676 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:16.503998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c208a-216b-477b-b97b-a05a02a68676 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:16.515498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a84d2e-6b84-4ba5-91a7-fc1fe72f8912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:16.516310 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a84d2e-6b84-4ba5-91a7-fc1fe72f8912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:16.525471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a84d2e-6b84-4ba5-91a7-fc1fe72f8912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:16.525766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a84d2e-6b84-4ba5-91a7-fc1fe72f8912 admin admin] PATH_INFO: `/v3/projects/804a9162166b4297808bcf72b6a2205e/users/c3cf9565e294425ba4ee5af464144754/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:16.576922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a84d2e-6b84-4ba5-91a7-fc1fe72f8912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=804a9162166b4297808bcf72b6a2205e, user_id=c3cf9565e294425ba4ee5af464144754, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:16.591873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a84d2e-6b84-4ba5-91a7-fc1fe72f8912 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:16.725335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 911/1813] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:16 2022] PUT /identity/v3/projects/804a9162166b4297808bcf72b6a2205e/users/c3cf9565e294425ba4ee5af464144754/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:16.742498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269885b6-e5f3-4522-b825-c6eda29b762b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:16.743085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269885b6-e5f3-4522-b825-c6eda29b762b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:16.743085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269885b6-e5f3-4522-b825-c6eda29b762b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:16.777214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-269885b6-e5f3-4522-b825-c6eda29b762b None None] MFA Rules not processed for user `c3cf9565e294425ba4ee5af464144754`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:16.778516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-269885b6-e5f3-4522-b825-c6eda29b762b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:16.857657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 903/1814] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:16.862959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a84d2e-6b84-4ba5-91a7-fc1fe72f8912 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:16.927860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a84d2e-6b84-4ba5-91a7-fc1fe72f8912 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:16.934427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be2b9a-d720-4a0c-9a70-b260e9c91b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:16.935046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be2b9a-d720-4a0c-9a70-b260e9c91b18 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:16.935333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be2b9a-d720-4a0c-9a70-b260e9c91b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:16.935596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be2b9a-d720-4a0c-9a70-b260e9c91b18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:16.936237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be2b9a-d720-4a0c-9a70-b260e9c91b18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:16.939114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be2b9a-d720-4a0c-9a70-b260e9c91b18 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:17.045179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 912/1815] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:16 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:00:17.046289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269885b6-e5f3-4522-b825-c6eda29b762b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:17.052952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:17.057389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269885b6-e5f3-4522-b825-c6eda29b762b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:17.115477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2499f06-f85f-47d2-86a8-4e0dcdb1c5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:17.116127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2499f06-f85f-47d2-86a8-4e0dcdb1c5a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:17.116411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2499f06-f85f-47d2-86a8-4e0dcdb1c5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:17.116688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2499f06-f85f-47d2-86a8-4e0dcdb1c5a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:17.117371 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2499f06-f85f-47d2-86a8-4e0dcdb1c5a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:17.119374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2499f06-f85f-47d2-86a8-4e0dcdb1c5a0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:17.302921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 904/1816] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:17 2022] POST /identity/v3/users => generated 300 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:00:17.304432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be2b9a-d720-4a0c-9a70-b260e9c91b18 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:17.320383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be2b9a-d720-4a0c-9a70-b260e9c91b18 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:17.335548 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda59bcb-9323-47e2-8919-53ef24f9478f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:17.336244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda59bcb-9323-47e2-8919-53ef24f9478f admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:17.337708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda59bcb-9323-47e2-8919-53ef24f9478f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:17.337708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda59bcb-9323-47e2-8919-53ef24f9478f admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:17.340357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda59bcb-9323-47e2-8919-53ef24f9478f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:17.340357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda59bcb-9323-47e2-8919-53ef24f9478f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:17.357052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 913/1817] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:17.363074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2499f06-f85f-47d2-86a8-4e0dcdb1c5a0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:17.379743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2499f06-f85f-47d2-86a8-4e0dcdb1c5a0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:17.386844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a3ed1-5fb3-4b1d-9db3-8100d2c0c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:17.390819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3ed1-5fb3-4b1d-9db3-8100d2c0c1de admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:17.390819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3ed1-5fb3-4b1d-9db3-8100d2c0c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:17.390819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3ed1-5fb3-4b1d-9db3-8100d2c0c1de admin admin] PATH_INFO: `/v3/projects/1e450a83cf764f74b150b733a4db6a86/users/52ae0411ff7346e0b8d25e5d1d3aae0e/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:17.410407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a3ed1-5fb3-4b1d-9db3-8100d2c0c1de admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e450a83cf764f74b150b733a4db6a86, user_id=52ae0411ff7346e0b8d25e5d1d3aae0e, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:17.413518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a3ed1-5fb3-4b1d-9db3-8100d2c0c1de admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:17.546105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 905/1818] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:17 2022] PUT /identity/v3/projects/1e450a83cf764f74b150b733a4db6a86/users/52ae0411ff7346e0b8d25e5d1d3aae0e/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:17.553336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda59bcb-9323-47e2-8919-53ef24f9478f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:17.565420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda59bcb-9323-47e2-8919-53ef24f9478f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:17.674123 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a9dad-feb8-4224-bb1a-b6545fd96ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:17.674945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a9dad-feb8-4224-bb1a-b6545fd96ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:17.675819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a9dad-feb8-4224-bb1a-b6545fd96ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:17.676195 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a9dad-feb8-4224-bb1a-b6545fd96ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:17.676954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a9dad-feb8-4224-bb1a-b6545fd96ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:17.678817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a9dad-feb8-4224-bb1a-b6545fd96ed1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:17.691862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 914/1819] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:17.695731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a3ed1-5fb3-4b1d-9db3-8100d2c0c1de admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:17.707775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a3ed1-5fb3-4b1d-9db3-8100d2c0c1de admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:17.713587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e989f24-af8e-4c42-8813-ddafede25f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:17.714259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e989f24-af8e-4c42-8813-ddafede25f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:17.714537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e989f24-af8e-4c42-8813-ddafede25f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:17.714813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e989f24-af8e-4c42-8813-ddafede25f56 admin admin] PATH_INFO: `/v3/projects/1e450a83cf764f74b150b733a4db6a86/users/52ae0411ff7346e0b8d25e5d1d3aae0e/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:17.716975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e989f24-af8e-4c42-8813-ddafede25f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e450a83cf764f74b150b733a4db6a86, user_id=52ae0411ff7346e0b8d25e5d1d3aae0e, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:17.719673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e989f24-af8e-4c42-8813-ddafede25f56 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:17.741064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 906/1820] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:17 2022] PUT /identity/v3/projects/1e450a83cf764f74b150b733a4db6a86/users/52ae0411ff7346e0b8d25e5d1d3aae0e/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:00:17.744376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c2492-e07c-4d95-b27c-ec4796cf2751 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:17.744781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c2492-e07c-4d95-b27c-ec4796cf2751 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:17.745178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c2492-e07c-4d95-b27c-ec4796cf2751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:17.779904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-810c2492-e07c-4d95-b27c-ec4796cf2751 None None] MFA Rules not processed for user `52ae0411ff7346e0b8d25e5d1d3aae0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:17.781270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-810c2492-e07c-4d95-b27c-ec4796cf2751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:17.815663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:17.919225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 915/1821] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:17.932309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e989f24-af8e-4c42-8813-ddafede25f56 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:17.934819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8e989f24-af8e-4c42-8813-ddafede25f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:18.014517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a167c2c-ea2e-42db-9f2c-4d2db21f1a03 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:18.015624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a167c2c-ea2e-42db-9f2c-4d2db21f1a03 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:18.016198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a167c2c-ea2e-42db-9f2c-4d2db21f1a03 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:18.018670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 916/1822] 10.222.0.17 () {54 vars in 921 bytes} [Thu Aug 11 10:00:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:00:18.033239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679940cf-e6f0-412a-8788-a445cd801fd6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:18.033551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679940cf-e6f0-412a-8788-a445cd801fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:18.033806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679940cf-e6f0-412a-8788-a445cd801fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:18.041668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e989f24-af8e-4c42-8813-ddafede25f56 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:18.045816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9053c-b8df-46f9-bc57-c1e29955f0f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:18.053190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9053c-b8df-46f9-bc57-c1e29955f0f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:18.053190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9053c-b8df-46f9-bc57-c1e29955f0f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:18.053190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9053c-b8df-46f9-bc57-c1e29955f0f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:18.058848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9053c-b8df-46f9-bc57-c1e29955f0f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:18.061364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9053c-b8df-46f9-bc57-c1e29955f0f0 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:18.072965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-679940cf-e6f0-412a-8788-a445cd801fd6 None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:18.075982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-679940cf-e6f0-412a-8788-a445cd801fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:18.091776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 907/1823] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:18.133146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9053c-b8df-46f9-bc57-c1e29955f0f0 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:18.208399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9053c-b8df-46f9-bc57-c1e29955f0f0 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:18.218808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:18.219592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:18.219953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:18.220298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] PATH_INFO: `/v3/users/c3cf9565e294425ba4ee5af464144754` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:18.223529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3cf9565e294425ba4ee5af464144754)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:18.225601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:18.234458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:18.273587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 917/1824] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Aug 11 10:00:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:18.307259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679940cf-e6f0-412a-8788-a445cd801fd6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:18.309859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-679940cf-e6f0-412a-8788-a445cd801fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:18.340479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679940cf-e6f0-412a-8788-a445cd801fd6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:18.348924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f8fc9-b272-480f-bc93-5ab76efe231f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:18.350500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f8fc9-b272-480f-bc93-5ab76efe231f service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:18.351115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f8fc9-b272-480f-bc93-5ab76efe231f service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:18.351525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f8fc9-b272-480f-bc93-5ab76efe231f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:18.364459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f8fc9-b272-480f-bc93-5ab76efe231f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:18.366838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f8fc9-b272-480f-bc93-5ab76efe231f service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:18.403128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 918/1825] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:00:18.416734 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3cf9565e294425ba4ee5af464144754'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:18.421935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3cf9565e294425ba4ee5af464144754'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:18.421935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3cf9565e294425ba4ee5af464144754'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:18.532424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3cf9565e294425ba4ee5af464144754'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:18.547772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3cf9565e294425ba4ee5af464144754'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:18.566710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 908/1826] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:18 2022] DELETE /identity/v3/users/c3cf9565e294425ba4ee5af464144754 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:18.571137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b7994-3115-4158-a487-e445d84224c1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:18.571603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b7994-3115-4158-a487-e445d84224c1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:18.571705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b7994-3115-4158-a487-e445d84224c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:18.613046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-646b7994-3115-4158-a487-e445d84224c1 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:18.614369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-646b7994-3115-4158-a487-e445d84224c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:18.727594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 919/1827] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:18.743433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:18.745521 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:18.863557 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204ff15-0ef1-4c74-9474-c1e4830a5ada admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:18.868217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c69b6-1ad6-42fa-bb87-6b3cf637bc99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:18.869277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c69b6-1ad6-42fa-bb87-6b3cf637bc99 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:18.869277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c69b6-1ad6-42fa-bb87-6b3cf637bc99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:18.869448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c69b6-1ad6-42fa-bb87-6b3cf637bc99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:18.886394 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c69b6-1ad6-42fa-bb87-6b3cf637bc99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:18.893616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c69b6-1ad6-42fa-bb87-6b3cf637bc99 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:18.904706 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 909/1828] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:19.780097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b7994-3115-4158-a487-e445d84224c1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:19.782755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-646b7994-3115-4158-a487-e445d84224c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:19.801996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b7994-3115-4158-a487-e445d84224c1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:19.803296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:19.804046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:19.804567 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:19.804950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] PATH_INFO: `/v3/projects/804a9162166b4297808bcf72b6a2205e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:19.806312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=804a9162166b4297808bcf72b6a2205e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:19.808671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:19.914797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:19.926803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '804a9162166b4297808bcf72b6a2205e', 'request_id': 'req-f690cb4a-59d2-4086-8c05-e3d9387b25d1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:19.932465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '804a9162166b4297808bcf72b6a2205e', 'request_id': 'req-f690cb4a-59d2-4086-8c05-e3d9387b25d1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:19.939557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '804a9162166b4297808bcf72b6a2205e', 'request_id': 'req-f690cb4a-59d2-4086-8c05-e3d9387b25d1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:20.006343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] The token cache is being invalidate because project 804a9162166b4297808bcf72b6a2205e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:20.006890 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:20.007993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:20.009608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 920/1829] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:19 2022] DELETE /identity/v3/projects/804a9162166b4297808bcf72b6a2205e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:20.013651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c69b6-1ad6-42fa-bb87-6b3cf637bc99 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:20.024293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c69b6-1ad6-42fa-bb87-6b3cf637bc99 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:20.088357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233c844-38b6-433f-95fb-17b8283961a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:20.089144 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233c844-38b6-433f-95fb-17b8283961a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:20.089437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233c844-38b6-433f-95fb-17b8283961a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:20.089719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233c844-38b6-433f-95fb-17b8283961a1 admin admin] PATH_INFO: `/v3/users/52ae0411ff7346e0b8d25e5d1d3aae0e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:20.091106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233c844-38b6-433f-95fb-17b8283961a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ae0411ff7346e0b8d25e5d1d3aae0e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:20.093791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233c844-38b6-433f-95fb-17b8283961a1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:20.121656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:20.287309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8233c844-38b6-433f-95fb-17b8283961a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ae0411ff7346e0b8d25e5d1d3aae0e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:20.288099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8233c844-38b6-433f-95fb-17b8283961a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ae0411ff7346e0b8d25e5d1d3aae0e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:20.288790 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8233c844-38b6-433f-95fb-17b8283961a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ae0411ff7346e0b8d25e5d1d3aae0e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:20.360934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8233c844-38b6-433f-95fb-17b8283961a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ae0411ff7346e0b8d25e5d1d3aae0e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:20.384271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8233c844-38b6-433f-95fb-17b8283961a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ae0411ff7346e0b8d25e5d1d3aae0e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:20.404329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 910/1830] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:20 2022] DELETE /identity/v3/users/52ae0411ff7346e0b8d25e5d1d3aae0e => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:21.095816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:21.097933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:21.123144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690cb4a-59d2-4086-8c05-e3d9387b25d1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:21.248104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:21.249870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:21.250758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:21.251305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] PATH_INFO: `/v3/projects/1e450a83cf764f74b150b733a4db6a86` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:21.253398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e450a83cf764f74b150b733a4db6a86)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:21.256970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:21.361207 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e450a83cf764f74b150b733a4db6a86', 'request_id': 'req-06c9e5ed-09af-4680-bc07-e6a39e56980d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:21.374066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e450a83cf764f74b150b733a4db6a86', 'request_id': 'req-06c9e5ed-09af-4680-bc07-e6a39e56980d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:21.381898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e450a83cf764f74b150b733a4db6a86', 'request_id': 'req-06c9e5ed-09af-4680-bc07-e6a39e56980d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:21.465152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] The token cache is being invalidate because project 1e450a83cf764f74b150b733a4db6a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:21.465444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:21.466085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-06c9e5ed-09af-4680-bc07-e6a39e56980d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:21.472174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 921/1831] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:21 2022] DELETE /identity/v3/projects/1e450a83cf764f74b150b733a4db6a86 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:21.473683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35313c35-3645-4c17-a6f2-49b75a37671a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:21.474131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35313c35-3645-4c17-a6f2-49b75a37671a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:21.474220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35313c35-3645-4c17-a6f2-49b75a37671a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:21.495691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-35313c35-3645-4c17-a6f2-49b75a37671a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:21.496917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-35313c35-3645-4c17-a6f2-49b75a37671a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:21.590640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 911/1832] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:21.591738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7bf4f-3b6a-4eda-af22-7f6523039796 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:21.591738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7bf4f-3b6a-4eda-af22-7f6523039796 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:21.591738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7bf4f-3b6a-4eda-af22-7f6523039796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:21.629973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f9b7bf4f-3b6a-4eda-af22-7f6523039796 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:21.631617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f9b7bf4f-3b6a-4eda-af22-7f6523039796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:21.644275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 922/1833] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:00:21.648684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35313c35-3645-4c17-a6f2-49b75a37671a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:21.661853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35313c35-3645-4c17-a6f2-49b75a37671a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:21.752068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d20e20-991f-43a6-937a-7c1957ec4860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:21.752068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d20e20-991f-43a6-937a-7c1957ec4860 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:21.752068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d20e20-991f-43a6-937a-7c1957ec4860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:21.752068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d20e20-991f-43a6-937a-7c1957ec4860 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:21.752406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d20e20-991f-43a6-937a-7c1957ec4860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:00:21.752406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d20e20-991f-43a6-937a-7c1957ec4860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:21.752406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d20e20-991f-43a6-937a-7c1957ec4860 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:21.752406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 912/1834] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:00:21.755853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7bf4f-3b6a-4eda-af22-7f6523039796 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:21.768774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7bf4f-3b6a-4eda-af22-7f6523039796 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:21.775464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a2973-c132-435e-a7d4-3bfe8e67229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:21.780675 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a2973-c132-435e-a7d4-3bfe8e67229c admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:21.781753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a2973-c132-435e-a7d4-3bfe8e67229c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:21.782224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a2973-c132-435e-a7d4-3bfe8e67229c admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:21.783073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a2973-c132-435e-a7d4-3bfe8e67229c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:21.785412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a2973-c132-435e-a7d4-3bfe8e67229c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:22.094798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 923/1835] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:21 2022] POST /identity/v3/projects => generated 367 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:22.096390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d20e20-991f-43a6-937a-7c1957ec4860 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:22.106792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d20e20-991f-43a6-937a-7c1957ec4860 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:22.188849 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515123f-7b1f-4f46-b2a8-2e35d2602065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:22.189597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515123f-7b1f-4f46-b2a8-2e35d2602065 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:22.189955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515123f-7b1f-4f46-b2a8-2e35d2602065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:22.192254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515123f-7b1f-4f46-b2a8-2e35d2602065 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:22.193255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515123f-7b1f-4f46-b2a8-2e35d2602065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:22.198074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515123f-7b1f-4f46-b2a8-2e35d2602065 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:22.307013 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 913/1836] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:22 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:22.310020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a2973-c132-435e-a7d4-3bfe8e67229c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:22.324704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a2973-c132-435e-a7d4-3bfe8e67229c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:22.333463 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a8634-006e-471f-b211-3d6e34dffa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:22.334099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8634-006e-471f-b211-3d6e34dffa04 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:22.334384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8634-006e-471f-b211-3d6e34dffa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:22.334657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a8634-006e-471f-b211-3d6e34dffa04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:22.335317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a8634-006e-471f-b211-3d6e34dffa04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:22.336917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a8634-006e-471f-b211-3d6e34dffa04 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:22.347546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 924/1837] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:22.352203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515123f-7b1f-4f46-b2a8-2e35d2602065 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:22.362799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515123f-7b1f-4f46-b2a8-2e35d2602065 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:22.368468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b4b9c-3a52-4fbb-acc6-90196b54cfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:22.369269 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b4b9c-3a52-4fbb-acc6-90196b54cfc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:22.372124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b4b9c-3a52-4fbb-acc6-90196b54cfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:22.372124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b4b9c-3a52-4fbb-acc6-90196b54cfc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:22.372124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b4b9c-3a52-4fbb-acc6-90196b54cfc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:22.372525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b4b9c-3a52-4fbb-acc6-90196b54cfc0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:22.384613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 914/1838] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:22.389159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a8634-006e-471f-b211-3d6e34dffa04 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:22.406562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a8634-006e-471f-b211-3d6e34dffa04 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:22.411727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ea190-807d-4095-a521-b8e60f0c5022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:22.412561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ea190-807d-4095-a521-b8e60f0c5022 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:22.412873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ea190-807d-4095-a521-b8e60f0c5022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:22.413205 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ea190-807d-4095-a521-b8e60f0c5022 admin admin] PATH_INFO: `/v3/projects/c6e91885bcc34f0b97a762ec564cfd88/users/6a677821eff44f54a6c05b2c888d65da/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:22.424718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:22.434372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ea190-807d-4095-a521-b8e60f0c5022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6e91885bcc34f0b97a762ec564cfd88, user_id=6a677821eff44f54a6c05b2c888d65da, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:22.440441 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ea190-807d-4095-a521-b8e60f0c5022 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:22.533131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 925/1839] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:22 2022] PUT /identity/v3/projects/c6e91885bcc34f0b97a762ec564cfd88/users/6a677821eff44f54a6c05b2c888d65da/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:22.540838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610850c-59de-4d25-b888-444cd0be64e8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:22.541406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610850c-59de-4d25-b888-444cd0be64e8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:22.541549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610850c-59de-4d25-b888-444cd0be64e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:22.561971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:22.581396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e610850c-59de-4d25-b888-444cd0be64e8 None None] MFA Rules not processed for user `6a677821eff44f54a6c05b2c888d65da`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:22.582206 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e610850c-59de-4d25-b888-444cd0be64e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:22.661960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 915/1840] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:22.662875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ea190-807d-4095-a521-b8e60f0c5022 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:22.731990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ea190-807d-4095-a521-b8e60f0c5022 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:22.736642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee067d9-aa08-47fa-a66e-fdf807bcddd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:22.737329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee067d9-aa08-47fa-a66e-fdf807bcddd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:22.737583 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee067d9-aa08-47fa-a66e-fdf807bcddd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:22.737883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee067d9-aa08-47fa-a66e-fdf807bcddd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:22.738545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee067d9-aa08-47fa-a66e-fdf807bcddd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:22.740386 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee067d9-aa08-47fa-a66e-fdf807bcddd0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:22.820280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 926/1841] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:22 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:00:22.824166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610850c-59de-4d25-b888-444cd0be64e8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:22.838342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610850c-59de-4d25-b888-444cd0be64e8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:22.900955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c75d3-8a7a-4a2e-8028-180c13f91976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:22.906138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c75d3-8a7a-4a2e-8028-180c13f91976 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:22.906625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c75d3-8a7a-4a2e-8028-180c13f91976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:22.906913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c75d3-8a7a-4a2e-8028-180c13f91976 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:22.907699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c75d3-8a7a-4a2e-8028-180c13f91976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:22.909669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c75d3-8a7a-4a2e-8028-180c13f91976 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:23.004854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 916/1842] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:22 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:00:23.008387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee067d9-aa08-47fa-a66e-fdf807bcddd0 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:23.020437 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee067d9-aa08-47fa-a66e-fdf807bcddd0 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:23.026531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ad379-2649-4f21-9c00-ad45e8e929fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:23.027209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad379-2649-4f21-9c00-ad45e8e929fa admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:23.027784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad379-2649-4f21-9c00-ad45e8e929fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:23.028064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad379-2649-4f21-9c00-ad45e8e929fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:23.028723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ad379-2649-4f21-9c00-ad45e8e929fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:23.031353 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ad379-2649-4f21-9c00-ad45e8e929fa admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:23.047686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 927/1843] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:23.054443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c75d3-8a7a-4a2e-8028-180c13f91976 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:23.070119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c75d3-8a7a-4a2e-8028-180c13f91976 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:23.076495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd6545-35bf-4e9b-8f55-08d2f8c0d28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:23.077255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd6545-35bf-4e9b-8f55-08d2f8c0d28d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:23.077694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd6545-35bf-4e9b-8f55-08d2f8c0d28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:23.077977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd6545-35bf-4e9b-8f55-08d2f8c0d28d admin admin] PATH_INFO: `/v3/projects/2ffa1ed5ec1b4ab697503f438933cd2e/users/32faf38f1e25426ab9876fa5491eefdf/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:23.099958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd6545-35bf-4e9b-8f55-08d2f8c0d28d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffa1ed5ec1b4ab697503f438933cd2e, user_id=32faf38f1e25426ab9876fa5491eefdf, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:23.102939 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd6545-35bf-4e9b-8f55-08d2f8c0d28d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:23.172572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 917/1844] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:23 2022] PUT /identity/v3/projects/2ffa1ed5ec1b4ab697503f438933cd2e/users/32faf38f1e25426ab9876fa5491eefdf/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:23.178897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ad379-2649-4f21-9c00-ad45e8e929fa admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:23.190666 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ad379-2649-4f21-9c00-ad45e8e929fa admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:23.275753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7e7a1-04df-4e26-aac8-a897d69ecbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:23.276396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e7a1-04df-4e26-aac8-a897d69ecbc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:23.276641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e7a1-04df-4e26-aac8-a897d69ecbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:23.276910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e7a1-04df-4e26-aac8-a897d69ecbc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:23.277610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7e7a1-04df-4e26-aac8-a897d69ecbc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:23.279198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7e7a1-04df-4e26-aac8-a897d69ecbc8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:23.293047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 928/1845] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:23.294713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd6545-35bf-4e9b-8f55-08d2f8c0d28d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:23.306731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd6545-35bf-4e9b-8f55-08d2f8c0d28d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:23.311747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8b993-c438-4a70-9c95-67ef70eaff56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:23.312415 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8b993-c438-4a70-9c95-67ef70eaff56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:23.312696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8b993-c438-4a70-9c95-67ef70eaff56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:23.312969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8b993-c438-4a70-9c95-67ef70eaff56 admin admin] PATH_INFO: `/v3/projects/2ffa1ed5ec1b4ab697503f438933cd2e/users/32faf38f1e25426ab9876fa5491eefdf/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:23.314942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8b993-c438-4a70-9c95-67ef70eaff56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffa1ed5ec1b4ab697503f438933cd2e, user_id=32faf38f1e25426ab9876fa5491eefdf, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:23.317607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8b993-c438-4a70-9c95-67ef70eaff56 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:23.330876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 918/1846] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:23 2022] PUT /identity/v3/projects/2ffa1ed5ec1b4ab697503f438933cd2e/users/32faf38f1e25426ab9876fa5491eefdf/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:00:23.343515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc20cc-71b7-4e51-92f3-8f85dbe972bb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:23.347818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc20cc-71b7-4e51-92f3-8f85dbe972bb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:23.347818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc20cc-71b7-4e51-92f3-8f85dbe972bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:23.374739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-30fc20cc-71b7-4e51-92f3-8f85dbe972bb None None] MFA Rules not processed for user `32faf38f1e25426ab9876fa5491eefdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:23.376053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-30fc20cc-71b7-4e51-92f3-8f85dbe972bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:23.427496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:23.452795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 929/1847] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:23.466204 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8b993-c438-4a70-9c95-67ef70eaff56 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:23.468002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0bc8b993-c438-4a70-9c95-67ef70eaff56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:23.531540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:23.540743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8b993-c438-4a70-9c95-67ef70eaff56 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:23.540966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a380f33-be06-4846-b902-05e866b4b73e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:23.541547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a380f33-be06-4846-b902-05e866b4b73e service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:23.541816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a380f33-be06-4846-b902-05e866b4b73e service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:23.542094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a380f33-be06-4846-b902-05e866b4b73e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:23.552882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a380f33-be06-4846-b902-05e866b4b73e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:23.554406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a380f33-be06-4846-b902-05e866b4b73e service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:23.565740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 919/1848] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:23.787539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc20cc-71b7-4e51-92f3-8f85dbe972bb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:23.797097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc20cc-71b7-4e51-92f3-8f85dbe972bb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:23.801066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898de96-33e3-4e15-9b66-01c5d36bcc65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:23.801683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898de96-33e3-4e15-9b66-01c5d36bcc65 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:23.801961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898de96-33e3-4e15-9b66-01c5d36bcc65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:23.802231 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898de96-33e3-4e15-9b66-01c5d36bcc65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:23.871926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898de96-33e3-4e15-9b66-01c5d36bcc65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:23.873535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898de96-33e3-4e15-9b66-01c5d36bcc65 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:23.884238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 930/1849] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:25.799768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a380f33-be06-4846-b902-05e866b4b73e service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:25.801879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7a380f33-be06-4846-b902-05e866b4b73e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:25.911981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a380f33-be06-4846-b902-05e866b4b73e service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:25.917104 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:25.918214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:25.918726 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:25.919348 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] PATH_INFO: `/v3/users/24fc56655181409ca2ecd89adc824450` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:25.921939 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24fc56655181409ca2ecd89adc824450)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:25.925188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:26.164899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24fc56655181409ca2ecd89adc824450'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:26.165606 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24fc56655181409ca2ecd89adc824450'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:26.166634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24fc56655181409ca2ecd89adc824450'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:26.262439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24fc56655181409ca2ecd89adc824450'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:26.281563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24fc56655181409ca2ecd89adc824450'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:26.297542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 920/1850] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:25 2022] DELETE /identity/v3/users/24fc56655181409ca2ecd89adc824450 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:26.301562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc0070-ea51-4a14-9fce-474451d4cb97 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:26.303066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc0070-ea51-4a14-9fce-474451d4cb97 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:26.303347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc0070-ea51-4a14-9fce-474451d4cb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:26.326145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-cedc0070-ea51-4a14-9fce-474451d4cb97 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:26.327536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cedc0070-ea51-4a14-9fce-474451d4cb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:26.443110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 931/1851] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:26.453556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:26.461757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:26.537129 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a547f-7b00-4621-a799-531fec641319 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:26.540798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406b4fb-e41d-45cd-8f0d-acb43e2d2e67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:26.541551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406b4fb-e41d-45cd-8f0d-acb43e2d2e67 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:26.541940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406b4fb-e41d-45cd-8f0d-acb43e2d2e67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:26.542826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406b4fb-e41d-45cd-8f0d-acb43e2d2e67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:26.553454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406b4fb-e41d-45cd-8f0d-acb43e2d2e67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:26.554903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406b4fb-e41d-45cd-8f0d-acb43e2d2e67 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:26.565163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 921/1852] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:27.040528 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc0070-ea51-4a14-9fce-474451d4cb97 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:27.044384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cedc0070-ea51-4a14-9fce-474451d4cb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:27.068348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc0070-ea51-4a14-9fce-474451d4cb97 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:27.073376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:27.074072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:27.074391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:27.074734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] PATH_INFO: `/v3/projects/b8d6cd1ee98d49b2a31cd74eec99ac7b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:27.076076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d6cd1ee98d49b2a31cd74eec99ac7b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:27.079346 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:27.321606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d6cd1ee98d49b2a31cd74eec99ac7b', 'request_id': 'req-2467f66e-9172-45fb-9aaf-3545cc95a608'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:27.328659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d6cd1ee98d49b2a31cd74eec99ac7b', 'request_id': 'req-2467f66e-9172-45fb-9aaf-3545cc95a608'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:27.338174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d6cd1ee98d49b2a31cd74eec99ac7b', 'request_id': 'req-2467f66e-9172-45fb-9aaf-3545cc95a608'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:27.423514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] The token cache is being invalidate because project b8d6cd1ee98d49b2a31cd74eec99ac7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:27.423810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:27.424460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:27.429210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 932/1853] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:27 2022] DELETE /identity/v3/projects/b8d6cd1ee98d49b2a31cd74eec99ac7b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:27.430336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406b4fb-e41d-45cd-8f0d-acb43e2d2e67 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:27.449106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406b4fb-e41d-45cd-8f0d-acb43e2d2e67 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:27.470394 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:27.561051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c559ce3-eafb-4873-9c48-2d3c9e8f872a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:27.562452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c559ce3-eafb-4873-9c48-2d3c9e8f872a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:27.565928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c559ce3-eafb-4873-9c48-2d3c9e8f872a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:27.565928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c559ce3-eafb-4873-9c48-2d3c9e8f872a admin admin] PATH_INFO: `/v3/users/264cf761ea7b4c9a9921edce8aeaceea` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:27.565928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c559ce3-eafb-4873-9c48-2d3c9e8f872a admin admin] RBAC: Authorizing `identity:delete_user(user_id=264cf761ea7b4c9a9921edce8aeaceea)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:27.568568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c559ce3-eafb-4873-9c48-2d3c9e8f872a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:27.775438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5c559ce3-eafb-4873-9c48-2d3c9e8f872a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '264cf761ea7b4c9a9921edce8aeaceea'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:27.776456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5c559ce3-eafb-4873-9c48-2d3c9e8f872a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '264cf761ea7b4c9a9921edce8aeaceea'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:27.777262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5c559ce3-eafb-4873-9c48-2d3c9e8f872a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '264cf761ea7b4c9a9921edce8aeaceea'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:27.903124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5c559ce3-eafb-4873-9c48-2d3c9e8f872a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '264cf761ea7b4c9a9921edce8aeaceea'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:27.924847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5c559ce3-eafb-4873-9c48-2d3c9e8f872a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '264cf761ea7b4c9a9921edce8aeaceea'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:27.938757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 922/1854] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:27 2022] DELETE /identity/v3/users/264cf761ea7b4c9a9921edce8aeaceea => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:28.538172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:28.539697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:28.550330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:28.554489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467f66e-9172-45fb-9aaf-3545cc95a608 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:28.623192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:28.624622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:28.625001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:28.625408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] PATH_INFO: `/v3/projects/482fd8bc50e74dd49c8f4231de1658cf` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:28.626806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=482fd8bc50e74dd49c8f4231de1658cf)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:28.629880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:28.785158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '482fd8bc50e74dd49c8f4231de1658cf', 'request_id': 'req-8dbac641-809d-4188-8383-90c3e0b4ace4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:28.791041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '482fd8bc50e74dd49c8f4231de1658cf', 'request_id': 'req-8dbac641-809d-4188-8383-90c3e0b4ace4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:28.803672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '482fd8bc50e74dd49c8f4231de1658cf', 'request_id': 'req-8dbac641-809d-4188-8383-90c3e0b4ace4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:28.900307 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] The token cache is being invalidate because project 482fd8bc50e74dd49c8f4231de1658cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:28.900499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:28.901340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dbac641-809d-4188-8383-90c3e0b4ace4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:28.903058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 933/1855] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:28 2022] DELETE /identity/v3/projects/482fd8bc50e74dd49c8f4231de1658cf => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:28.909540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9221e-da89-4cff-a201-934f302e29d9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:28.910360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9221e-da89-4cff-a201-934f302e29d9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:28.910810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9221e-da89-4cff-a201-934f302e29d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:28.932196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fec9221e-da89-4cff-a201-934f302e29d9 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:28.933639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fec9221e-da89-4cff-a201-934f302e29d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:29.002616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 923/1856] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:29.010458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21d1eb-1c9d-40c5-8762-fb883e81c4d9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:29.010744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21d1eb-1c9d-40c5-8762-fb883e81c4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:29.011021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21d1eb-1c9d-40c5-8762-fb883e81c4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:29.030771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5e21d1eb-1c9d-40c5-8762-fb883e81c4d9 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:29.031969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5e21d1eb-1c9d-40c5-8762-fb883e81c4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:29.039649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 934/1857] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:29.043792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9221e-da89-4cff-a201-934f302e29d9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:29.054068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9221e-da89-4cff-a201-934f302e29d9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:29.058691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674c6a4-6054-4479-a8c5-18ba2ad2bbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:29.059292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674c6a4-6054-4479-a8c5-18ba2ad2bbeb admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:29.059578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674c6a4-6054-4479-a8c5-18ba2ad2bbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:29.059854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674c6a4-6054-4479-a8c5-18ba2ad2bbeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:29.060526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674c6a4-6054-4479-a8c5-18ba2ad2bbeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:00:29.060840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674c6a4-6054-4479-a8c5-18ba2ad2bbeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:29.063281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674c6a4-6054-4479-a8c5-18ba2ad2bbeb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:29.079318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 924/1858] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:00:29.080127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21d1eb-1c9d-40c5-8762-fb883e81c4d9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:29.090065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21d1eb-1c9d-40c5-8762-fb883e81c4d9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:29.095241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2c7e0-2989-4f9b-be68-e7fa6a010827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:29.095856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2c7e0-2989-4f9b-be68-e7fa6a010827 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:29.096141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2c7e0-2989-4f9b-be68-e7fa6a010827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:29.096416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2c7e0-2989-4f9b-be68-e7fa6a010827 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:29.097085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2c7e0-2989-4f9b-be68-e7fa6a010827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:29.098988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2c7e0-2989-4f9b-be68-e7fa6a010827 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:29.190053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 935/1859] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:29 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:29.194380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674c6a4-6054-4479-a8c5-18ba2ad2bbeb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:29.206009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674c6a4-6054-4479-a8c5-18ba2ad2bbeb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:29.223371 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:29.265625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663f3f0-ced5-4470-8a99-6f90d658565b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:29.266260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663f3f0-ced5-4470-8a99-6f90d658565b admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:29.266633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663f3f0-ced5-4470-8a99-6f90d658565b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:29.266807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663f3f0-ced5-4470-8a99-6f90d658565b admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:29.267444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663f3f0-ced5-4470-8a99-6f90d658565b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:29.269356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663f3f0-ced5-4470-8a99-6f90d658565b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:29.354403 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 925/1860] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:29 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:00:29.359150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2c7e0-2989-4f9b-be68-e7fa6a010827 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:29.369937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2c7e0-2989-4f9b-be68-e7fa6a010827 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:29.378027 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd060293-1713-4e8a-87e9-3430571b2682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:29.378027 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd060293-1713-4e8a-87e9-3430571b2682 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:29.378027 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd060293-1713-4e8a-87e9-3430571b2682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:29.378027 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd060293-1713-4e8a-87e9-3430571b2682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:29.378382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd060293-1713-4e8a-87e9-3430571b2682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:29.379232 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd060293-1713-4e8a-87e9-3430571b2682 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:29.392874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 936/1861] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:29.394198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663f3f0-ced5-4470-8a99-6f90d658565b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:29.404450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663f3f0-ced5-4470-8a99-6f90d658565b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:29.409575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da758c8c-23ba-4477-af4f-a20f31e88525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:29.410184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da758c8c-23ba-4477-af4f-a20f31e88525 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:29.410466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da758c8c-23ba-4477-af4f-a20f31e88525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:29.410739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da758c8c-23ba-4477-af4f-a20f31e88525 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:29.411382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da758c8c-23ba-4477-af4f-a20f31e88525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:29.412956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da758c8c-23ba-4477-af4f-a20f31e88525 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:29.424863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 926/1862] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:29.429070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd060293-1713-4e8a-87e9-3430571b2682 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:29.439861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd060293-1713-4e8a-87e9-3430571b2682 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:29.444451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd40de8-a069-4ea1-8adb-0fe4878e5791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:29.445184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd40de8-a069-4ea1-8adb-0fe4878e5791 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:29.445466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd40de8-a069-4ea1-8adb-0fe4878e5791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:29.445770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd40de8-a069-4ea1-8adb-0fe4878e5791 admin admin] PATH_INFO: `/v3/projects/bc3f147489504a128366e9f8e040f80b/users/947b1f3755424fd7aca3d8111471f854/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:29.460885 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd40de8-a069-4ea1-8adb-0fe4878e5791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3f147489504a128366e9f8e040f80b, user_id=947b1f3755424fd7aca3d8111471f854, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:29.463824 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd40de8-a069-4ea1-8adb-0fe4878e5791 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:29.566751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 937/1863] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:29 2022] PUT /identity/v3/projects/bc3f147489504a128366e9f8e040f80b/users/947b1f3755424fd7aca3d8111471f854/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:00:29.577215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825e6ae-9135-48cf-8063-4a1cd93b1395 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:29.577773 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825e6ae-9135-48cf-8063-4a1cd93b1395 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:29.578134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825e6ae-9135-48cf-8063-4a1cd93b1395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:29.609264 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0825e6ae-9135-48cf-8063-4a1cd93b1395 None None] MFA Rules not processed for user `947b1f3755424fd7aca3d8111471f854`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:29.610876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0825e6ae-9135-48cf-8063-4a1cd93b1395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:29.681024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 927/1864] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:29.685472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd40de8-a069-4ea1-8adb-0fe4878e5791 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:29.721704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:29.752517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd40de8-a069-4ea1-8adb-0fe4878e5791 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:29.757288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b950f-0775-4166-a914-65f414a540cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:29.757901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b950f-0775-4166-a914-65f414a540cf admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:29.758185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b950f-0775-4166-a914-65f414a540cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:29.760206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b950f-0775-4166-a914-65f414a540cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:29.760206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b950f-0775-4166-a914-65f414a540cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:29.760946 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656b950f-0775-4166-a914-65f414a540cf admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:29.845087 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 938/1865] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:29 2022] POST /identity/v3/projects => generated 387 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:29.846182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825e6ae-9135-48cf-8063-4a1cd93b1395 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:29.857085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825e6ae-9135-48cf-8063-4a1cd93b1395 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:29.914728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d6686-b52d-4ce6-a8d4-342edfa2e9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:29.915374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d6686-b52d-4ce6-a8d4-342edfa2e9ba admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:29.915610 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d6686-b52d-4ce6-a8d4-342edfa2e9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:29.915882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d6686-b52d-4ce6-a8d4-342edfa2e9ba admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:29.916520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d6686-b52d-4ce6-a8d4-342edfa2e9ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:29.918442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d6686-b52d-4ce6-a8d4-342edfa2e9ba admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:30.003104 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 928/1866] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:29 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:30.004739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b950f-0775-4166-a914-65f414a540cf admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:30.016426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b950f-0775-4166-a914-65f414a540cf admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:30.021226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a1993-7948-41f7-bbe3-dbb74dbec76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:30.021907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a1993-7948-41f7-bbe3-dbb74dbec76d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:30.022196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a1993-7948-41f7-bbe3-dbb74dbec76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:30.022495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a1993-7948-41f7-bbe3-dbb74dbec76d admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:30.023143 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a1993-7948-41f7-bbe3-dbb74dbec76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:30.024737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a1993-7948-41f7-bbe3-dbb74dbec76d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:30.036227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 939/1867] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:30.040514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d6686-b52d-4ce6-a8d4-342edfa2e9ba admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:30.052092 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d6686-b52d-4ce6-a8d4-342edfa2e9ba admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:30.058253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00403d99-2d62-4ff8-b3c1-41b5a97a8fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:30.058920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00403d99-2d62-4ff8-b3c1-41b5a97a8fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:30.059200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00403d99-2d62-4ff8-b3c1-41b5a97a8fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:30.059474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00403d99-2d62-4ff8-b3c1-41b5a97a8fcb admin admin] PATH_INFO: `/v3/projects/94793f68e5314e169533a2102e13c04a/users/6ba8dcdd9aec4079a512a0b61e93e410/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:30.074833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00403d99-2d62-4ff8-b3c1-41b5a97a8fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=94793f68e5314e169533a2102e13c04a, user_id=6ba8dcdd9aec4079a512a0b61e93e410, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:30.078082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00403d99-2d62-4ff8-b3c1-41b5a97a8fcb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:30.080362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:30.138187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 929/1868] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:30 2022] PUT /identity/v3/projects/94793f68e5314e169533a2102e13c04a/users/6ba8dcdd9aec4079a512a0b61e93e410/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:00:30.142308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a1993-7948-41f7-bbe3-dbb74dbec76d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:30.152294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a1993-7948-41f7-bbe3-dbb74dbec76d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:30.211908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff9cd7-ca78-49f2-99a7-9bb7a4a06042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:30.212779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff9cd7-ca78-49f2-99a7-9bb7a4a06042 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:30.213092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff9cd7-ca78-49f2-99a7-9bb7a4a06042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:30.213366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff9cd7-ca78-49f2-99a7-9bb7a4a06042 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:30.214009 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff9cd7-ca78-49f2-99a7-9bb7a4a06042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:30.215589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff9cd7-ca78-49f2-99a7-9bb7a4a06042 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:30.225502 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 940/1869] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:00:30.229709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00403d99-2d62-4ff8-b3c1-41b5a97a8fcb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:30.240417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00403d99-2d62-4ff8-b3c1-41b5a97a8fcb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:30.245103 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ed476-d4e3-48ef-8e21-feb90782a40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:30.245758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ed476-d4e3-48ef-8e21-feb90782a40a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:30.246038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ed476-d4e3-48ef-8e21-feb90782a40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:30.246307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ed476-d4e3-48ef-8e21-feb90782a40a admin admin] PATH_INFO: `/v3/projects/94793f68e5314e169533a2102e13c04a/users/6ba8dcdd9aec4079a512a0b61e93e410/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:30.248207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ed476-d4e3-48ef-8e21-feb90782a40a admin admin] RBAC: Authorizing `identity:create_grant(project_id=94793f68e5314e169533a2102e13c04a, user_id=6ba8dcdd9aec4079a512a0b61e93e410, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:30.251127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ed476-d4e3-48ef-8e21-feb90782a40a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:30.261524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 930/1870] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:30 2022] PUT /identity/v3/projects/94793f68e5314e169533a2102e13c04a/users/6ba8dcdd9aec4079a512a0b61e93e410/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:00:30.273135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b7447-d29a-4418-811c-9fe5d51641c0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:30.273423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b7447-d29a-4418-811c-9fe5d51641c0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:30.273696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b7447-d29a-4418-811c-9fe5d51641c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:30.301872 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-290b7447-d29a-4418-811c-9fe5d51641c0 None None] MFA Rules not processed for user `6ba8dcdd9aec4079a512a0b61e93e410`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:30.303090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-290b7447-d29a-4418-811c-9fe5d51641c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:30.373252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 941/1871] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:30.395382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ed476-d4e3-48ef-8e21-feb90782a40a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:30.397101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f95ed476-d4e3-48ef-8e21-feb90782a40a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:30.457471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ed476-d4e3-48ef-8e21-feb90782a40a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:30.461517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d2f5c-990a-4f2e-908d-07285956cf13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:30.462140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d2f5c-990a-4f2e-908d-07285956cf13 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:30.462426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d2f5c-990a-4f2e-908d-07285956cf13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:30.463200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d2f5c-990a-4f2e-908d-07285956cf13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:30.475367 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d2f5c-990a-4f2e-908d-07285956cf13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:30.476801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d2f5c-990a-4f2e-908d-07285956cf13 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:30.487238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 931/1872] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:00:30.633539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b7447-d29a-4418-811c-9fe5d51641c0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:30.668546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:30.718325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b7447-d29a-4418-811c-9fe5d51641c0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:30.723872 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:30.724770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:30.725078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:30.725435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] PATH_INFO: `/v3/users/947b1f3755424fd7aca3d8111471f854` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:30.726579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=947b1f3755424fd7aca3d8111471f854)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:30.728966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:30.911090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947b1f3755424fd7aca3d8111471f854'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:30.918410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947b1f3755424fd7aca3d8111471f854'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:30.984090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947b1f3755424fd7aca3d8111471f854'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:30.984752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947b1f3755424fd7aca3d8111471f854'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:30.998049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947b1f3755424fd7aca3d8111471f854'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:31.005341 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 942/1873] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:30 2022] DELETE /identity/v3/users/947b1f3755424fd7aca3d8111471f854 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:31.009913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8cff8-d12c-47ac-b891-df9453473b2e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:31.010735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8cff8-d12c-47ac-b891-df9453473b2e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:31.011018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8cff8-d12c-47ac-b891-df9453473b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:31.031201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e3e8cff8-d12c-47ac-b891-df9453473b2e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:31.032358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e3e8cff8-d12c-47ac-b891-df9453473b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:31.095144 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 932/1874] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:31.105547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:31.107052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:31.173311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae4c73-be23-4430-b585-0089bdcd82bd admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:31.176926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f8127-8753-44a5-b51c-d301d1cd37c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:31.177565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f8127-8753-44a5-b51c-d301d1cd37c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:31.177846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f8127-8753-44a5-b51c-d301d1cd37c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:31.178121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f8127-8753-44a5-b51c-d301d1cd37c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:31.188946 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f8127-8753-44a5-b51c-d301d1cd37c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:31.189993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f8127-8753-44a5-b51c-d301d1cd37c4 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:31.201061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 943/1875] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:00:31.636083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8cff8-d12c-47ac-b891-df9453473b2e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:31.639691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e3e8cff8-d12c-47ac-b891-df9453473b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:31.653719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8cff8-d12c-47ac-b891-df9453473b2e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:31.665443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:31.666108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:31.666389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:31.666669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] PATH_INFO: `/v3/projects/bc3f147489504a128366e9f8e040f80b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:31.667812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3f147489504a128366e9f8e040f80b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:31.671498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:31.778789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3f147489504a128366e9f8e040f80b', 'request_id': 'req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:31.784989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3f147489504a128366e9f8e040f80b', 'request_id': 'req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:31.791270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3f147489504a128366e9f8e040f80b', 'request_id': 'req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:31.841460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] The token cache is being invalidate because project bc3f147489504a128366e9f8e040f80b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:31.841765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:31.842606 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:31.848023 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 933/1876] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:31 2022] DELETE /identity/v3/projects/bc3f147489504a128366e9f8e040f80b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:31.848753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f8127-8753-44a5-b51c-d301d1cd37c4 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:31.860493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f8127-8753-44a5-b51c-d301d1cd37c4 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:31.915987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:31.928844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c62e98-90aa-4814-a772-f5e4c9d11747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:31.929523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c62e98-90aa-4814-a772-f5e4c9d11747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:31.929809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c62e98-90aa-4814-a772-f5e4c9d11747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:31.930213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c62e98-90aa-4814-a772-f5e4c9d11747 admin admin] PATH_INFO: `/v3/users/6ba8dcdd9aec4079a512a0b61e93e410` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:31.931407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c62e98-90aa-4814-a772-f5e4c9d11747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ba8dcdd9aec4079a512a0b61e93e410)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:31.933366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c62e98-90aa-4814-a772-f5e4c9d11747 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:32.097666 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-89c62e98-90aa-4814-a772-f5e4c9d11747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ba8dcdd9aec4079a512a0b61e93e410'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:32.105407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-89c62e98-90aa-4814-a772-f5e4c9d11747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ba8dcdd9aec4079a512a0b61e93e410'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:32.148861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-89c62e98-90aa-4814-a772-f5e4c9d11747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ba8dcdd9aec4079a512a0b61e93e410'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:32.149527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-89c62e98-90aa-4814-a772-f5e4c9d11747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ba8dcdd9aec4079a512a0b61e93e410'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:32.163648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-89c62e98-90aa-4814-a772-f5e4c9d11747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ba8dcdd9aec4079a512a0b61e93e410'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:32.169955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 944/1877] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:31 2022] DELETE /identity/v3/users/6ba8dcdd9aec4079a512a0b61e93e410 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:00:32.786364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:32.788568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:32.807268 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1daaa27-4bd4-4b1c-8f48-f664e1638f5f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:32.813489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:32.873295 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:32.874119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:32.874395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:32.874668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] PATH_INFO: `/v3/projects/94793f68e5314e169533a2102e13c04a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:32.875842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94793f68e5314e169533a2102e13c04a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:32.877822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:33.005709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94793f68e5314e169533a2102e13c04a', 'request_id': 'req-8d219215-7be9-4fcd-b088-fecbce1e0ac7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:33.011030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94793f68e5314e169533a2102e13c04a', 'request_id': 'req-8d219215-7be9-4fcd-b088-fecbce1e0ac7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:33.018185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94793f68e5314e169533a2102e13c04a', 'request_id': 'req-8d219215-7be9-4fcd-b088-fecbce1e0ac7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:33.074231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] The token cache is being invalidate because project 94793f68e5314e169533a2102e13c04a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:33.074526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:33.075188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8d219215-7be9-4fcd-b088-fecbce1e0ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:33.077630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 934/1878] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:32 2022] DELETE /identity/v3/projects/94793f68e5314e169533a2102e13c04a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:33.082497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee21687-9805-4564-92e0-4a72437f76cd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:33.083009 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee21687-9805-4564-92e0-4a72437f76cd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:33.083126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee21687-9805-4564-92e0-4a72437f76cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:33.105309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-fee21687-9805-4564-92e0-4a72437f76cd None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:33.107062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-fee21687-9805-4564-92e0-4a72437f76cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:33.178901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 945/1879] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:33.186818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4b245-30cb-44da-95b8-a23f1deb8a2c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:33.187100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4b245-30cb-44da-95b8-a23f1deb8a2c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:33.187376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4b245-30cb-44da-95b8-a23f1deb8a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:33.213445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f2d4b245-30cb-44da-95b8-a23f1deb8a2c None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:33.214884 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f2d4b245-30cb-44da-95b8-a23f1deb8a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:33.225455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 935/1880] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:33.229843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee21687-9805-4564-92e0-4a72437f76cd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:33.240559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee21687-9805-4564-92e0-4a72437f76cd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:33.245362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1961906-920b-4b75-9ec3-83350f55c0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:33.245973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961906-920b-4b75-9ec3-83350f55c0ae admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:33.246254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961906-920b-4b75-9ec3-83350f55c0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:33.246528 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961906-920b-4b75-9ec3-83350f55c0ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:33.247208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1961906-920b-4b75-9ec3-83350f55c0ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:00:33.247509 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1961906-920b-4b75-9ec3-83350f55c0ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:33.249236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1961906-920b-4b75-9ec3-83350f55c0ae admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:33.261211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 946/1881] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:00:33.266131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4b245-30cb-44da-95b8-a23f1deb8a2c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:33.275566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4b245-30cb-44da-95b8-a23f1deb8a2c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:33.281223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ba73c-1ec6-4e16-beaa-f7a8f64e8c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:33.281844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ba73c-1ec6-4e16-beaa-f7a8f64e8c4a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:33.282110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ba73c-1ec6-4e16-beaa-f7a8f64e8c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:33.282382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ba73c-1ec6-4e16-beaa-f7a8f64e8c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:33.283020 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ba73c-1ec6-4e16-beaa-f7a8f64e8c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:33.284903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ba73c-1ec6-4e16-beaa-f7a8f64e8c4a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:33.360858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 936/1882] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:33 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:33.365138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1961906-920b-4b75-9ec3-83350f55c0ae admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:33.375334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1961906-920b-4b75-9ec3-83350f55c0ae admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:33.426731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:33.433494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c304c-b8e5-4228-b0fd-699018e98d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:33.434088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c304c-b8e5-4228-b0fd-699018e98d8d admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:33.434364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c304c-b8e5-4228-b0fd-699018e98d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:33.434633 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c304c-b8e5-4228-b0fd-699018e98d8d admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:33.435257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c304c-b8e5-4228-b0fd-699018e98d8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:33.437911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c304c-b8e5-4228-b0fd-699018e98d8d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:33.588673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 947/1883] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:33 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:33.592903 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ba73c-1ec6-4e16-beaa-f7a8f64e8c4a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:33.603272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ba73c-1ec6-4e16-beaa-f7a8f64e8c4a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:33.608162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a811f-d8b2-4a36-a1e7-5422c9daa9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:33.608785 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a811f-d8b2-4a36-a1e7-5422c9daa9ca admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:33.609085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a811f-d8b2-4a36-a1e7-5422c9daa9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:33.609359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a811f-d8b2-4a36-a1e7-5422c9daa9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:33.610046 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a811f-d8b2-4a36-a1e7-5422c9daa9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:33.611641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a811f-d8b2-4a36-a1e7-5422c9daa9ca admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:33.625145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 937/1884] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:33.626442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c304c-b8e5-4228-b0fd-699018e98d8d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:33.636595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c304c-b8e5-4228-b0fd-699018e98d8d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:33.641852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114e2e7-d3d7-44e6-964b-be449ee080b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:33.642453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114e2e7-d3d7-44e6-964b-be449ee080b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:33.642736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114e2e7-d3d7-44e6-964b-be449ee080b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:33.643012 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114e2e7-d3d7-44e6-964b-be449ee080b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:33.643654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114e2e7-d3d7-44e6-964b-be449ee080b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:33.645274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114e2e7-d3d7-44e6-964b-be449ee080b6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:33.656008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 948/1885] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:33.660128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a811f-d8b2-4a36-a1e7-5422c9daa9ca admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:33.670014 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a811f-d8b2-4a36-a1e7-5422c9daa9ca admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:33.674701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062eb58e-6410-4dbe-8ab4-27681189daab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:33.675360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062eb58e-6410-4dbe-8ab4-27681189daab admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:33.675646 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062eb58e-6410-4dbe-8ab4-27681189daab admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:33.675925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062eb58e-6410-4dbe-8ab4-27681189daab admin admin] PATH_INFO: `/v3/projects/9feaf190f0f34a5fb3c5b2c072368939/users/35bb3a93c22140939d9e15b5167c0d97/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:33.690909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062eb58e-6410-4dbe-8ab4-27681189daab admin admin] RBAC: Authorizing `identity:create_grant(project_id=9feaf190f0f34a5fb3c5b2c072368939, user_id=35bb3a93c22140939d9e15b5167c0d97, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:33.693514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062eb58e-6410-4dbe-8ab4-27681189daab admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:33.759633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 938/1886] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:33 2022] PUT /identity/v3/projects/9feaf190f0f34a5fb3c5b2c072368939/users/35bb3a93c22140939d9e15b5167c0d97/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:33.772142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcae2-646b-4cd6-ae1a-aba2339b46c8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:33.772429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcae2-646b-4cd6-ae1a-aba2339b46c8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:33.772711 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcae2-646b-4cd6-ae1a-aba2339b46c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:33.800066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-016bcae2-646b-4cd6-ae1a-aba2339b46c8 None None] MFA Rules not processed for user `35bb3a93c22140939d9e15b5167c0d97`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:33.801315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-016bcae2-646b-4cd6-ae1a-aba2339b46c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:33.863962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 949/1887] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:33.868432 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062eb58e-6410-4dbe-8ab4-27681189daab admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:33.902472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:33.930398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062eb58e-6410-4dbe-8ab4-27681189daab admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:33.935107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af634a-c7fd-4146-8bf5-a76629fb0d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:33.935737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af634a-c7fd-4146-8bf5-a76629fb0d97 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:33.936950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af634a-c7fd-4146-8bf5-a76629fb0d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:33.937478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af634a-c7fd-4146-8bf5-a76629fb0d97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:33.938411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af634a-c7fd-4146-8bf5-a76629fb0d97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:33.940844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af634a-c7fd-4146-8bf5-a76629fb0d97 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:34.030641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 939/1888] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:33 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:34.034661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bcae2-646b-4cd6-ae1a-aba2339b46c8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:34.045896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bcae2-646b-4cd6-ae1a-aba2339b46c8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:34.101254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0e438-de30-4e25-bc6c-13d723beb637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:34.101851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0e438-de30-4e25-bc6c-13d723beb637 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:34.102131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0e438-de30-4e25-bc6c-13d723beb637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:34.102391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0e438-de30-4e25-bc6c-13d723beb637 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:34.103019 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0e438-de30-4e25-bc6c-13d723beb637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:34.104916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0e438-de30-4e25-bc6c-13d723beb637 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:34.189220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 950/1889] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:34 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:00:34.192978 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af634a-c7fd-4146-8bf5-a76629fb0d97 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:34.203819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af634a-c7fd-4146-8bf5-a76629fb0d97 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:34.208466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5768c2-e9a9-4df3-a911-1a24a589ee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:34.209097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5768c2-e9a9-4df3-a911-1a24a589ee4c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:34.209377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5768c2-e9a9-4df3-a911-1a24a589ee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:34.209648 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5768c2-e9a9-4df3-a911-1a24a589ee4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:34.210312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5768c2-e9a9-4df3-a911-1a24a589ee4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:34.211876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5768c2-e9a9-4df3-a911-1a24a589ee4c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:34.222944 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 940/1890] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:34.226740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0e438-de30-4e25-bc6c-13d723beb637 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:34.237665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0e438-de30-4e25-bc6c-13d723beb637 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:34.242901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c0659-a98b-4da7-a6bc-5ef5f34d5514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:34.243514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c0659-a98b-4da7-a6bc-5ef5f34d5514 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:34.243817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c0659-a98b-4da7-a6bc-5ef5f34d5514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:34.244072 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c0659-a98b-4da7-a6bc-5ef5f34d5514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:34.244727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c0659-a98b-4da7-a6bc-5ef5f34d5514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:34.246385 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c0659-a98b-4da7-a6bc-5ef5f34d5514 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:34.267341 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 951/1891] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:34.267489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5768c2-e9a9-4df3-a911-1a24a589ee4c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:34.272692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5768c2-e9a9-4df3-a911-1a24a589ee4c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:34.277567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f7921-46d6-4481-919d-504c374ee69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:34.278305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f7921-46d6-4481-919d-504c374ee69a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:34.278589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f7921-46d6-4481-919d-504c374ee69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:34.278864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f7921-46d6-4481-919d-504c374ee69a admin admin] PATH_INFO: `/v3/projects/78071a594be4426b83fb46785425eb8f/users/55808a0c5482417cbc8ffcb7cdeef110/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:34.294495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f7921-46d6-4481-919d-504c374ee69a admin admin] RBAC: Authorizing `identity:create_grant(project_id=78071a594be4426b83fb46785425eb8f, user_id=55808a0c5482417cbc8ffcb7cdeef110, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:34.297931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f7921-46d6-4481-919d-504c374ee69a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:34.355008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 941/1892] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:34 2022] PUT /identity/v3/projects/78071a594be4426b83fb46785425eb8f/users/55808a0c5482417cbc8ffcb7cdeef110/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:00:34.367024 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68fd42-580a-4061-8464-f1c05ec0cdf4 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:34.367546 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68fd42-580a-4061-8464-f1c05ec0cdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:34.367928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68fd42-580a-4061-8464-f1c05ec0cdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:34.396963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1d68fd42-580a-4061-8464-f1c05ec0cdf4 None None] MFA Rules not processed for user `55808a0c5482417cbc8ffcb7cdeef110`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:34.398313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1d68fd42-580a-4061-8464-f1c05ec0cdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:34.439364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:34.465650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 952/1893] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:34.470642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f7921-46d6-4481-919d-504c374ee69a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:34.535467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f7921-46d6-4481-919d-504c374ee69a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:34.540190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e883a7e-e6c4-4690-a826-dff5d64606ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:34.540853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e883a7e-e6c4-4690-a826-dff5d64606ea admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:34.541172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e883a7e-e6c4-4690-a826-dff5d64606ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:34.541448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e883a7e-e6c4-4690-a826-dff5d64606ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:34.542083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e883a7e-e6c4-4690-a826-dff5d64606ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:34.543979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e883a7e-e6c4-4690-a826-dff5d64606ea admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:34.651880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 942/1894] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:34 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:00:34.653558 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68fd42-580a-4061-8464-f1c05ec0cdf4 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:34.664870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68fd42-580a-4061-8464-f1c05ec0cdf4 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:34.728539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49b090-06b8-4837-9132-870ef89ff42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:34.729174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49b090-06b8-4837-9132-870ef89ff42c admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:34.729460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49b090-06b8-4837-9132-870ef89ff42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:34.729738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49b090-06b8-4837-9132-870ef89ff42c admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:34.730356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49b090-06b8-4837-9132-870ef89ff42c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:34.732263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b49b090-06b8-4837-9132-870ef89ff42c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:34.800733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 953/1895] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:34 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:34.804728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e883a7e-e6c4-4690-a826-dff5d64606ea admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:34.815805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e883a7e-e6c4-4690-a826-dff5d64606ea admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:34.820373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0849fad-767d-4dfc-95ee-188a5aa7777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:34.820966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0849fad-767d-4dfc-95ee-188a5aa7777b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:34.821280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0849fad-767d-4dfc-95ee-188a5aa7777b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:34.821550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0849fad-767d-4dfc-95ee-188a5aa7777b admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:34.822187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0849fad-767d-4dfc-95ee-188a5aa7777b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:34.823741 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0849fad-767d-4dfc-95ee-188a5aa7777b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:34.834820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 943/1896] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:34.839204 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49b090-06b8-4837-9132-870ef89ff42c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:34.850740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49b090-06b8-4837-9132-870ef89ff42c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:34.855970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6ccf6-6219-47fa-9ca8-99b08466a41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:34.856635 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6ccf6-6219-47fa-9ca8-99b08466a41c admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:34.856910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6ccf6-6219-47fa-9ca8-99b08466a41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:34.857214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6ccf6-6219-47fa-9ca8-99b08466a41c admin admin] PATH_INFO: `/v3/projects/1ec208d999174bda9f47a6332174f28e/users/54bd89d8ed364f0ebcb84e74991ba956/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:34.874056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6ccf6-6219-47fa-9ca8-99b08466a41c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec208d999174bda9f47a6332174f28e, user_id=54bd89d8ed364f0ebcb84e74991ba956, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:34.876749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6ccf6-6219-47fa-9ca8-99b08466a41c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:34.955008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 954/1897] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:34 2022] PUT /identity/v3/projects/1ec208d999174bda9f47a6332174f28e/users/54bd89d8ed364f0ebcb84e74991ba956/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:34.959489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0849fad-767d-4dfc-95ee-188a5aa7777b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:34.971501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0849fad-767d-4dfc-95ee-188a5aa7777b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:35.025535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:35.032145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb98d8-d071-4102-87c0-28eda8592284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:35.032756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb98d8-d071-4102-87c0-28eda8592284 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:35.033058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb98d8-d071-4102-87c0-28eda8592284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:35.033338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb98d8-d071-4102-87c0-28eda8592284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:35.033988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb98d8-d071-4102-87c0-28eda8592284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:35.035595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb98d8-d071-4102-87c0-28eda8592284 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:35.046432 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 944/1898] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:35.050795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6ccf6-6219-47fa-9ca8-99b08466a41c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:35.061216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6ccf6-6219-47fa-9ca8-99b08466a41c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:35.065875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8869f2-778b-4c11-ba14-7f0119297dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:35.067850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8869f2-778b-4c11-ba14-7f0119297dd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:35.068127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8869f2-778b-4c11-ba14-7f0119297dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:35.068399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8869f2-778b-4c11-ba14-7f0119297dd8 admin admin] PATH_INFO: `/v3/projects/1ec208d999174bda9f47a6332174f28e/users/54bd89d8ed364f0ebcb84e74991ba956/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:35.070343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8869f2-778b-4c11-ba14-7f0119297dd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec208d999174bda9f47a6332174f28e, user_id=54bd89d8ed364f0ebcb84e74991ba956, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:35.073715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8869f2-778b-4c11-ba14-7f0119297dd8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:35.089137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 955/1899] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:35 2022] PUT /identity/v3/projects/1ec208d999174bda9f47a6332174f28e/users/54bd89d8ed364f0ebcb84e74991ba956/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:35.101756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f31bc-81ca-4b76-b923-cc80cfb77e10 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:35.102039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f31bc-81ca-4b76-b923-cc80cfb77e10 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:35.102314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f31bc-81ca-4b76-b923-cc80cfb77e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:35.142653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-854f31bc-81ca-4b76-b923-cc80cfb77e10 None None] MFA Rules not processed for user `54bd89d8ed364f0ebcb84e74991ba956`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:35.143963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-854f31bc-81ca-4b76-b923-cc80cfb77e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:35.215498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 945/1900] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:35.227486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8869f2-778b-4c11-ba14-7f0119297dd8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:35.229220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3d8869f2-778b-4c11-ba14-7f0119297dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:35.298066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8869f2-778b-4c11-ba14-7f0119297dd8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:35.303187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c212877-8ab7-48c1-84fb-2acb29dd4406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:35.304158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c212877-8ab7-48c1-84fb-2acb29dd4406 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:35.304616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c212877-8ab7-48c1-84fb-2acb29dd4406 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:35.305106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c212877-8ab7-48c1-84fb-2acb29dd4406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:35.316628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c212877-8ab7-48c1-84fb-2acb29dd4406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:35.318139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c212877-8ab7-48c1-84fb-2acb29dd4406 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:35.328703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 956/1901] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:35.432638 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f31bc-81ca-4b76-b923-cc80cfb77e10 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:35.434151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-854f31bc-81ca-4b76-b923-cc80cfb77e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:35.447297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f31bc-81ca-4b76-b923-cc80cfb77e10 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:35.450977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ea181-a49e-46dc-b43b-25adac8f9baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:35.452262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ea181-a49e-46dc-b43b-25adac8f9baa service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:35.452639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ea181-a49e-46dc-b43b-25adac8f9baa service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:35.452916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ea181-a49e-46dc-b43b-25adac8f9baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:35.515784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ea181-a49e-46dc-b43b-25adac8f9baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:35.517457 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ea181-a49e-46dc-b43b-25adac8f9baa service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:35.528523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 946/1902] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:38.405157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c212877-8ab7-48c1-84fb-2acb29dd4406 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:38.407408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5c212877-8ab7-48c1-84fb-2acb29dd4406 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:38.439904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:38.475310 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c212877-8ab7-48c1-84fb-2acb29dd4406 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:38.480214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:38.480853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:38.481177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:38.481455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] PATH_INFO: `/v3/users/6a677821eff44f54a6c05b2c888d65da` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:38.483177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a677821eff44f54a6c05b2c888d65da)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:38.485626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:38.664965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a677821eff44f54a6c05b2c888d65da'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:38.672283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a677821eff44f54a6c05b2c888d65da'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:38.725705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a677821eff44f54a6c05b2c888d65da'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:38.726334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a677821eff44f54a6c05b2c888d65da'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:38.738838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a677821eff44f54a6c05b2c888d65da'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:38.744662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 957/1903] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:38 2022] DELETE /identity/v3/users/6a677821eff44f54a6c05b2c888d65da => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:38.750194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f784c-fdce-4711-b15a-df68420b4975 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:38.750658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f784c-fdce-4711-b15a-df68420b4975 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:38.753227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f784c-fdce-4711-b15a-df68420b4975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:38.770390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5a5f784c-fdce-4711-b15a-df68420b4975 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:38.771563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5a5f784c-fdce-4711-b15a-df68420b4975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:38.840538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 947/1904] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:38.851013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:38.852565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:38.912508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fb775-a6a9-4eaa-aaa4-933f95ed335a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:38.916265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f6444-4dfb-4d8c-b9c3-17c0b3ccb50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:38.916889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f6444-4dfb-4d8c-b9c3-17c0b3ccb50a service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:38.917211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f6444-4dfb-4d8c-b9c3-17c0b3ccb50a service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:38.917481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f6444-4dfb-4d8c-b9c3-17c0b3ccb50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:38.927288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f6444-4dfb-4d8c-b9c3-17c0b3ccb50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:38.928696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f6444-4dfb-4d8c-b9c3-17c0b3ccb50a service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:38.939330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 958/1905] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:39.379589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f784c-fdce-4711-b15a-df68420b4975 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:39.381060 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5a5f784c-fdce-4711-b15a-df68420b4975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:39.396473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f784c-fdce-4711-b15a-df68420b4975 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:39.401694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:39.402713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:39.404071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:39.405162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] PATH_INFO: `/v3/projects/c6e91885bcc34f0b97a762ec564cfd88` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:39.406458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6e91885bcc34f0b97a762ec564cfd88)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:39.408422 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:39.425499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:39.532969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6e91885bcc34f0b97a762ec564cfd88', 'request_id': 'req-d3e42235-5b89-494f-a3f1-103cdad446dc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:39.538498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6e91885bcc34f0b97a762ec564cfd88', 'request_id': 'req-d3e42235-5b89-494f-a3f1-103cdad446dc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:39.545810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6e91885bcc34f0b97a762ec564cfd88', 'request_id': 'req-d3e42235-5b89-494f-a3f1-103cdad446dc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:39.639213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] The token cache is being invalidate because project c6e91885bcc34f0b97a762ec564cfd88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:39.639557 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:39.640365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:39.643585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 948/1906] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:39 2022] DELETE /identity/v3/projects/c6e91885bcc34f0b97a762ec564cfd88 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:39.651388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f6444-4dfb-4d8c-b9c3-17c0b3ccb50a service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:39.664336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f6444-4dfb-4d8c-b9c3-17c0b3ccb50a service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:39.713645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:39.726346 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fa12c-b2cd-4d00-a922-b6ca4dbf9995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:39.726998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fa12c-b2cd-4d00-a922-b6ca4dbf9995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:39.727275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fa12c-b2cd-4d00-a922-b6ca4dbf9995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:39.727555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fa12c-b2cd-4d00-a922-b6ca4dbf9995 admin admin] PATH_INFO: `/v3/users/32faf38f1e25426ab9876fa5491eefdf` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:39.728748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fa12c-b2cd-4d00-a922-b6ca4dbf9995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32faf38f1e25426ab9876fa5491eefdf)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:39.730797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fa12c-b2cd-4d00-a922-b6ca4dbf9995 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:39.934475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d6fa12c-b2cd-4d00-a922-b6ca4dbf9995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32faf38f1e25426ab9876fa5491eefdf'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:39.941911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d6fa12c-b2cd-4d00-a922-b6ca4dbf9995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32faf38f1e25426ab9876fa5491eefdf'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:39.994320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d6fa12c-b2cd-4d00-a922-b6ca4dbf9995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32faf38f1e25426ab9876fa5491eefdf'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:39.994947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d6fa12c-b2cd-4d00-a922-b6ca4dbf9995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32faf38f1e25426ab9876fa5491eefdf'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:40.007442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d6fa12c-b2cd-4d00-a922-b6ca4dbf9995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32faf38f1e25426ab9876fa5491eefdf'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:40.013352 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 959/1907] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:39 2022] DELETE /identity/v3/users/32faf38f1e25426ab9876fa5491eefdf => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:40.458550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:40.460608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:40.474578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e42235-5b89-494f-a3f1-103cdad446dc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:40.538720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:40.539358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:40.539640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:40.539914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] PATH_INFO: `/v3/projects/2ffa1ed5ec1b4ab697503f438933cd2e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:40.541056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffa1ed5ec1b4ab697503f438933cd2e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:40.542940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:40.658172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffa1ed5ec1b4ab697503f438933cd2e', 'request_id': 'req-dc746558-da0e-4197-894f-9a470e31d1de'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:40.664556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffa1ed5ec1b4ab697503f438933cd2e', 'request_id': 'req-dc746558-da0e-4197-894f-9a470e31d1de'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:40.672043 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffa1ed5ec1b4ab697503f438933cd2e', 'request_id': 'req-dc746558-da0e-4197-894f-9a470e31d1de'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:40.737639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] The token cache is being invalidate because project 2ffa1ed5ec1b4ab697503f438933cd2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:40.737932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:40.738580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-dc746558-da0e-4197-894f-9a470e31d1de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:40.740096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 949/1908] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:40 2022] DELETE /identity/v3/projects/2ffa1ed5ec1b4ab697503f438933cd2e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:40.745294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f457-46ac-4a9b-a818-9919827e9d47 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:40.745766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f457-46ac-4a9b-a818-9919827e9d47 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:40.745873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f457-46ac-4a9b-a818-9919827e9d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:40.766234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0f04f457-46ac-4a9b-a818-9919827e9d47 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:40.767446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0f04f457-46ac-4a9b-a818-9919827e9d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:40.828965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 960/1909] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:40.836795 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6b0ee-c4d3-4cab-bb33-eaf534565f01 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:40.837122 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6b0ee-c4d3-4cab-bb33-eaf534565f01 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:40.837405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6b0ee-c4d3-4cab-bb33-eaf534565f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:40.859267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-57c6b0ee-c4d3-4cab-bb33-eaf534565f01 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:40.860470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-57c6b0ee-c4d3-4cab-bb33-eaf534565f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:40.868277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 950/1910] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:40.873704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04f457-46ac-4a9b-a818-9919827e9d47 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:40.883875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04f457-46ac-4a9b-a818-9919827e9d47 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:40.888391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730c2f-8754-4cd2-98ce-e8f2355212a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:40.888971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730c2f-8754-4cd2-98ce-e8f2355212a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:40.889482 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730c2f-8754-4cd2-98ce-e8f2355212a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:40.889777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1730c2f-8754-4cd2-98ce-e8f2355212a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:40.890423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1730c2f-8754-4cd2-98ce-e8f2355212a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:00:40.890723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1730c2f-8754-4cd2-98ce-e8f2355212a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:40.892296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1730c2f-8754-4cd2-98ce-e8f2355212a5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:40.908652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 961/1911] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:00:40.913002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6b0ee-c4d3-4cab-bb33-eaf534565f01 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:40.923523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6b0ee-c4d3-4cab-bb33-eaf534565f01 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:40.930667 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c66b45-2ba0-4df4-9747-2d3b7b9c315c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:40.931611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c66b45-2ba0-4df4-9747-2d3b7b9c315c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:40.931897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c66b45-2ba0-4df4-9747-2d3b7b9c315c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:40.932171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c66b45-2ba0-4df4-9747-2d3b7b9c315c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:40.933230 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c66b45-2ba0-4df4-9747-2d3b7b9c315c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:40.935113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c66b45-2ba0-4df4-9747-2d3b7b9c315c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:41.024024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 951/1912] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:40 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:00:41.028091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730c2f-8754-4cd2-98ce-e8f2355212a5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:41.038965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1730c2f-8754-4cd2-98ce-e8f2355212a5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:41.088867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:41.096192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764d23b-a1bb-44cc-9ac7-ded540d75564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:41.096874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d23b-a1bb-44cc-9ac7-ded540d75564 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:41.097209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d23b-a1bb-44cc-9ac7-ded540d75564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:41.097479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d23b-a1bb-44cc-9ac7-ded540d75564 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:41.101446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764d23b-a1bb-44cc-9ac7-ded540d75564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:41.101446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764d23b-a1bb-44cc-9ac7-ded540d75564 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:41.182427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 962/1913] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:41 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:41.186607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c66b45-2ba0-4df4-9747-2d3b7b9c315c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:41.192171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:41.197863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c66b45-2ba0-4df4-9747-2d3b7b9c315c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:41.202654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9173036-280a-4342-957a-c4e1b37ed4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:41.204820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9173036-280a-4342-957a-c4e1b37ed4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:41.204820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9173036-280a-4342-957a-c4e1b37ed4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:41.204820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9173036-280a-4342-957a-c4e1b37ed4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:41.204820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9173036-280a-4342-957a-c4e1b37ed4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:41.206151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9173036-280a-4342-957a-c4e1b37ed4c6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:41.217585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 952/1914] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:41.221802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764d23b-a1bb-44cc-9ac7-ded540d75564 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:41.231691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764d23b-a1bb-44cc-9ac7-ded540d75564 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:41.236959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25cfe88-44ad-456f-8ad4-56781308e961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:41.237584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cfe88-44ad-456f-8ad4-56781308e961 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:41.237868 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cfe88-44ad-456f-8ad4-56781308e961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:41.238135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25cfe88-44ad-456f-8ad4-56781308e961 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:41.238856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cfe88-44ad-456f-8ad4-56781308e961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:41.240490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25cfe88-44ad-456f-8ad4-56781308e961 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:41.250956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 963/1915] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:41.254963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9173036-280a-4342-957a-c4e1b37ed4c6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:41.265684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9173036-280a-4342-957a-c4e1b37ed4c6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:41.270648 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e53c4b-4727-4e59-b528-18962b5e3a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:41.271307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e53c4b-4727-4e59-b528-18962b5e3a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:41.271583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e53c4b-4727-4e59-b528-18962b5e3a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:41.271854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e53c4b-4727-4e59-b528-18962b5e3a6a admin admin] PATH_INFO: `/v3/projects/5c553633f5654aed9c03c9b716083603/users/08c32b0e61c244d6b49ac467b6cde77e/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:41.286579 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e53c4b-4727-4e59-b528-18962b5e3a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c553633f5654aed9c03c9b716083603, user_id=08c32b0e61c244d6b49ac467b6cde77e, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:41.289215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e53c4b-4727-4e59-b528-18962b5e3a6a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:41.360967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 953/1916] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:41 2022] PUT /identity/v3/projects/5c553633f5654aed9c03c9b716083603/users/08c32b0e61c244d6b49ac467b6cde77e/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:41.372475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b79bd5-4510-4338-8eb0-5586221c8103 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:41.372762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b79bd5-4510-4338-8eb0-5586221c8103 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:41.373785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b79bd5-4510-4338-8eb0-5586221c8103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:41.401820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-10b79bd5-4510-4338-8eb0-5586221c8103 None None] MFA Rules not processed for user `08c32b0e61c244d6b49ac467b6cde77e`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:41.403136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-10b79bd5-4510-4338-8eb0-5586221c8103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:41.470447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 964/1917] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:41.481566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e53c4b-4727-4e59-b528-18962b5e3a6a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:41.483847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-58e53c4b-4727-4e59-b528-18962b5e3a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:41.540073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e53c4b-4727-4e59-b528-18962b5e3a6a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:41.543774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba16c3-5667-423f-bd04-d7d806e1383d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:41.544381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba16c3-5667-423f-bd04-d7d806e1383d service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:41.544661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba16c3-5667-423f-bd04-d7d806e1383d service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:41.545483 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba16c3-5667-423f-bd04-d7d806e1383d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:41.553616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba16c3-5667-423f-bd04-d7d806e1383d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:41.555017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba16c3-5667-423f-bd04-d7d806e1383d service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:41.564690 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 954/1918] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:45.049493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b79bd5-4510-4338-8eb0-5586221c8103 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:45.050955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-10b79bd5-4510-4338-8eb0-5586221c8103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:45.116031 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b79bd5-4510-4338-8eb0-5586221c8103 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:45.120552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:45.121214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:45.121500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:45.121776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] PATH_INFO: `/v3/users/35bb3a93c22140939d9e15b5167c0d97` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:45.122903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] RBAC: Authorizing `identity:delete_user(user_id=35bb3a93c22140939d9e15b5167c0d97)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:45.125429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:45.295230 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35bb3a93c22140939d9e15b5167c0d97'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:45.315337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35bb3a93c22140939d9e15b5167c0d97'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:45.388698 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35bb3a93c22140939d9e15b5167c0d97'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:45.389340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35bb3a93c22140939d9e15b5167c0d97'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:45.392198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:45.403114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35bb3a93c22140939d9e15b5167c0d97'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:45.409472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 965/1919] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:45 2022] DELETE /identity/v3/users/35bb3a93c22140939d9e15b5167c0d97 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:45.413950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080eecd1-d39f-445f-970c-e63ccbe778aa None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:45.414408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080eecd1-d39f-445f-970c-e63ccbe778aa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:45.414498 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080eecd1-d39f-445f-970c-e63ccbe778aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:45.434217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-080eecd1-d39f-445f-970c-e63ccbe778aa None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:45.435425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-080eecd1-d39f-445f-970c-e63ccbe778aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:45.495191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 955/1920] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:45.505623 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:45.507085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:45.567226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb09496-a46c-4135-962e-d6fb54a9a74f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:45.571037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fee4a-25e3-425f-8c05-7d13f4e93f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:45.571700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fee4a-25e3-425f-8c05-7d13f4e93f7e service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:45.571980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fee4a-25e3-425f-8c05-7d13f4e93f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:45.572253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fee4a-25e3-425f-8c05-7d13f4e93f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:45.581562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fee4a-25e3-425f-8c05-7d13f4e93f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:45.583086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fee4a-25e3-425f-8c05-7d13f4e93f7e service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:45.593067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 966/1921] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:46.118263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080eecd1-d39f-445f-970c-e63ccbe778aa None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:46.120068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-080eecd1-d39f-445f-970c-e63ccbe778aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:46.134959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080eecd1-d39f-445f-970c-e63ccbe778aa None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:46.140559 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:46.142056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:46.142056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:46.142317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] PATH_INFO: `/v3/projects/9feaf190f0f34a5fb3c5b2c072368939` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:46.143474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] RBAC: Authorizing `identity:delete_project(project_id=9feaf190f0f34a5fb3c5b2c072368939)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:46.145559 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:46.252956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9feaf190f0f34a5fb3c5b2c072368939', 'request_id': 'req-7142916f-9250-4c35-97cf-40765ad6e6af'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:46.258652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9feaf190f0f34a5fb3c5b2c072368939', 'request_id': 'req-7142916f-9250-4c35-97cf-40765ad6e6af'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:46.266674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9feaf190f0f34a5fb3c5b2c072368939', 'request_id': 'req-7142916f-9250-4c35-97cf-40765ad6e6af'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:46.268187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:46.392649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] The token cache is being invalidate because project 9feaf190f0f34a5fb3c5b2c072368939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:46.392914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:46.393704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:46.395848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 956/1922] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:46 2022] DELETE /identity/v3/projects/9feaf190f0f34a5fb3c5b2c072368939 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:46.399897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fee4a-25e3-425f-8c05-7d13f4e93f7e service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:46.409684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fee4a-25e3-425f-8c05-7d13f4e93f7e service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:46.470922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:46.471575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:46.471884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:46.472188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] PATH_INFO: `/v3/users/55808a0c5482417cbc8ffcb7cdeef110` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:46.473465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55808a0c5482417cbc8ffcb7cdeef110)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:46.475788 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:46.722062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55808a0c5482417cbc8ffcb7cdeef110'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:46.729211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55808a0c5482417cbc8ffcb7cdeef110'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:46.828606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55808a0c5482417cbc8ffcb7cdeef110'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:46.829276 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55808a0c5482417cbc8ffcb7cdeef110'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:46.842118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55808a0c5482417cbc8ffcb7cdeef110'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:46.848603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 967/1923] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:46 2022] DELETE /identity/v3/users/55808a0c5482417cbc8ffcb7cdeef110 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:00:47.253826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:47.255308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:47.269748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142916f-9250-4c35-97cf-40765ad6e6af admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:47.331659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:47.332301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:47.332583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:47.332859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] PATH_INFO: `/v3/projects/78071a594be4426b83fb46785425eb8f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:47.334028 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78071a594be4426b83fb46785425eb8f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:47.335930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:47.435084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78071a594be4426b83fb46785425eb8f', 'request_id': 'req-c3a69ebb-0032-4835-8c5e-97c9429d7404'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:47.441320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78071a594be4426b83fb46785425eb8f', 'request_id': 'req-c3a69ebb-0032-4835-8c5e-97c9429d7404'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:47.449128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78071a594be4426b83fb46785425eb8f', 'request_id': 'req-c3a69ebb-0032-4835-8c5e-97c9429d7404'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:47.536445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] The token cache is being invalidate because project 78071a594be4426b83fb46785425eb8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:47.536732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:47.537592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:47.539179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 957/1924] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:47 2022] DELETE /identity/v3/projects/78071a594be4426b83fb46785425eb8f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:47.543274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:47.544690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:47.554145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:47.557903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab3077-87c0-4144-bd95-adbf04f2cfe8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:47.613955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d8ab3-ff27-4c0d-a341-edc2334c572c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:47.614581 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d8ab3-ff27-4c0d-a341-edc2334c572c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:47.614870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d8ab3-ff27-4c0d-a341-edc2334c572c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:47.615145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d8ab3-ff27-4c0d-a341-edc2334c572c admin admin] PATH_INFO: `/v3/users/54bd89d8ed364f0ebcb84e74991ba956` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:47.616376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d8ab3-ff27-4c0d-a341-edc2334c572c admin admin] RBAC: Authorizing `identity:delete_user(user_id=54bd89d8ed364f0ebcb84e74991ba956)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:47.618381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d8ab3-ff27-4c0d-a341-edc2334c572c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:47.762272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c72d8ab3-ff27-4c0d-a341-edc2334c572c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54bd89d8ed364f0ebcb84e74991ba956'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:47.769133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c72d8ab3-ff27-4c0d-a341-edc2334c572c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54bd89d8ed364f0ebcb84e74991ba956'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:47.828343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c72d8ab3-ff27-4c0d-a341-edc2334c572c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54bd89d8ed364f0ebcb84e74991ba956'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:47.829318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c72d8ab3-ff27-4c0d-a341-edc2334c572c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54bd89d8ed364f0ebcb84e74991ba956'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:47.841851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-c72d8ab3-ff27-4c0d-a341-edc2334c572c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54bd89d8ed364f0ebcb84e74991ba956'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:47.848000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 968/1925] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:47 2022] DELETE /identity/v3/users/54bd89d8ed364f0ebcb84e74991ba956 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:48.230130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:48.231662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:48.247602 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a69ebb-0032-4835-8c5e-97c9429d7404 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:48.311818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:48.312491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:48.312819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:48.313126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] PATH_INFO: `/v3/projects/1ec208d999174bda9f47a6332174f28e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:48.314282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ec208d999174bda9f47a6332174f28e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:48.316485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:48.442020 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:48.459260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ec208d999174bda9f47a6332174f28e', 'request_id': 'req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:48.465576 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ec208d999174bda9f47a6332174f28e', 'request_id': 'req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:48.473154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ec208d999174bda9f47a6332174f28e', 'request_id': 'req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:48.576244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] The token cache is being invalidate because project 1ec208d999174bda9f47a6332174f28e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:48.576594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:48.577259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-19e9f3f3-c1cd-4eb6-af48-3561b11654a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:48.578805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 958/1926] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:48 2022] DELETE /identity/v3/projects/1ec208d999174bda9f47a6332174f28e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:48.585292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7397a-826c-4347-a48d-0f4480988c00 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:48.585779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7397a-826c-4347-a48d-0f4480988c00 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:48.585945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7397a-826c-4347-a48d-0f4480988c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:48.606466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4dd7397a-826c-4347-a48d-0f4480988c00 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:48.607670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4dd7397a-826c-4347-a48d-0f4480988c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:48.665325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 969/1927] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:48.674873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8fd3-905c-4759-8d56-06b1e5c2ad88 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:48.675159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8fd3-905c-4759-8d56-06b1e5c2ad88 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:48.675438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8fd3-905c-4759-8d56-06b1e5c2ad88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:48.694657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-899d8fd3-905c-4759-8d56-06b1e5c2ad88 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:48.695897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-899d8fd3-905c-4759-8d56-06b1e5c2ad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:48.704017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 959/1928] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:48.708895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7397a-826c-4347-a48d-0f4480988c00 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:48.718896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7397a-826c-4347-a48d-0f4480988c00 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:48.724213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898481a-0549-4ac3-bf9a-5fea8df017b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:48.724856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898481a-0549-4ac3-bf9a-5fea8df017b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:48.725180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898481a-0549-4ac3-bf9a-5fea8df017b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:48.725456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898481a-0549-4ac3-bf9a-5fea8df017b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:48.726107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898481a-0549-4ac3-bf9a-5fea8df017b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:00:48.726406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898481a-0549-4ac3-bf9a-5fea8df017b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:48.728315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898481a-0549-4ac3-bf9a-5fea8df017b6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:48.740121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 970/1929] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:00:48.744495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d8fd3-905c-4759-8d56-06b1e5c2ad88 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:48.754360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d8fd3-905c-4759-8d56-06b1e5c2ad88 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:48.758936 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c9178-a2d4-4bca-8400-9110ac84bdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:48.759533 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c9178-a2d4-4bca-8400-9110ac84bdd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:48.759812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c9178-a2d4-4bca-8400-9110ac84bdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:48.760115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c9178-a2d4-4bca-8400-9110ac84bdd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:48.760747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c9178-a2d4-4bca-8400-9110ac84bdd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:48.762640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c9178-a2d4-4bca-8400-9110ac84bdd0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:48.845204 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 960/1930] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:48 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:48.852084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898481a-0549-4ac3-bf9a-5fea8df017b6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:48.865745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898481a-0549-4ac3-bf9a-5fea8df017b6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:48.873733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:48.921740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0e2c2-e199-42b5-91f0-32423965f331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:48.922341 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0e2c2-e199-42b5-91f0-32423965f331 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:48.922623 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0e2c2-e199-42b5-91f0-32423965f331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:48.922896 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0e2c2-e199-42b5-91f0-32423965f331 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:48.923628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0e2c2-e199-42b5-91f0-32423965f331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:48.926104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0e2c2-e199-42b5-91f0-32423965f331 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:49.057356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 971/1931] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:48 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:49.065625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c9178-a2d4-4bca-8400-9110ac84bdd0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:49.075705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c9178-a2d4-4bca-8400-9110ac84bdd0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:49.080460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f6b0-dd37-4454-94bc-a2430caa1225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:49.081814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f6b0-dd37-4454-94bc-a2430caa1225 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:49.082093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f6b0-dd37-4454-94bc-a2430caa1225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:49.082363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6f6b0-dd37-4454-94bc-a2430caa1225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:49.083007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6f6b0-dd37-4454-94bc-a2430caa1225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:49.084610 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6f6b0-dd37-4454-94bc-a2430caa1225 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:49.097610 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 961/1932] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:49.098614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0e2c2-e199-42b5-91f0-32423965f331 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:49.108489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0e2c2-e199-42b5-91f0-32423965f331 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:49.113805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3207a49-34c2-434c-a4b7-ecca8a85d95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:49.114397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3207a49-34c2-434c-a4b7-ecca8a85d95f admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:49.114679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3207a49-34c2-434c-a4b7-ecca8a85d95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:49.114953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3207a49-34c2-434c-a4b7-ecca8a85d95f admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:49.115601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3207a49-34c2-434c-a4b7-ecca8a85d95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:49.117214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3207a49-34c2-434c-a4b7-ecca8a85d95f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:49.132730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 972/1933] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:00:49.136885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f6b0-dd37-4454-94bc-a2430caa1225 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:49.144301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6f6b0-dd37-4454-94bc-a2430caa1225 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:49.149454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae2ea0-43e9-4420-ba28-3b5479d2825b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:49.150109 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae2ea0-43e9-4420-ba28-3b5479d2825b admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:49.150392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae2ea0-43e9-4420-ba28-3b5479d2825b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:49.150667 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae2ea0-43e9-4420-ba28-3b5479d2825b admin admin] PATH_INFO: `/v3/projects/609b16aaf0b348c28d722baa78049f12/users/03d8f78baad548c98e16f3ede0f88c0d/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:49.166696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae2ea0-43e9-4420-ba28-3b5479d2825b admin admin] RBAC: Authorizing `identity:create_grant(project_id=609b16aaf0b348c28d722baa78049f12, user_id=03d8f78baad548c98e16f3ede0f88c0d, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:49.169604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae2ea0-43e9-4420-ba28-3b5479d2825b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:49.245881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 962/1934] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:49 2022] PUT /identity/v3/projects/609b16aaf0b348c28d722baa78049f12/users/03d8f78baad548c98e16f3ede0f88c0d/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:49.257693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203fe79-3f5a-4b1d-b3eb-d37b0ef8091d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:49.258002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203fe79-3f5a-4b1d-b3eb-d37b0ef8091d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:49.258272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203fe79-3f5a-4b1d-b3eb-d37b0ef8091d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:49.287857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1203fe79-3f5a-4b1d-b3eb-d37b0ef8091d None None] MFA Rules not processed for user `03d8f78baad548c98e16f3ede0f88c0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:49.289181 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1203fe79-3f5a-4b1d-b3eb-d37b0ef8091d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:49.358453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 973/1935] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:49.363697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae2ea0-43e9-4420-ba28-3b5479d2825b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:49.428301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae2ea0-43e9-4420-ba28-3b5479d2825b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:49.433474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0871e8-f197-4f1e-9029-c3cd56dad74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:49.434091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0871e8-f197-4f1e-9029-c3cd56dad74c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:49.434373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0871e8-f197-4f1e-9029-c3cd56dad74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:49.434645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0871e8-f197-4f1e-9029-c3cd56dad74c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:49.435281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0871e8-f197-4f1e-9029-c3cd56dad74c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:49.437239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0871e8-f197-4f1e-9029-c3cd56dad74c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:49.561801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 963/1936] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:49 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:00:49.563259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203fe79-3f5a-4b1d-b3eb-d37b0ef8091d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:49.578662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203fe79-3f5a-4b1d-b3eb-d37b0ef8091d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:49.641138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69022615-987c-48ac-9f41-08c04d4baf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:49.641822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69022615-987c-48ac-9f41-08c04d4baf0a admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:49.642075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69022615-987c-48ac-9f41-08c04d4baf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:49.642260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69022615-987c-48ac-9f41-08c04d4baf0a admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:49.642932 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69022615-987c-48ac-9f41-08c04d4baf0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:49.645217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69022615-987c-48ac-9f41-08c04d4baf0a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:49.724907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 974/1937] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:49 2022] POST /identity/v3/users => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:49.726153 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0871e8-f197-4f1e-9029-c3cd56dad74c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:49.737352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0871e8-f197-4f1e-9029-c3cd56dad74c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:49.742574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59051c-b5be-46e0-8e28-32d96d40377c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:49.743192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59051c-b5be-46e0-8e28-32d96d40377c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:49.743469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59051c-b5be-46e0-8e28-32d96d40377c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:49.743739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59051c-b5be-46e0-8e28-32d96d40377c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:49.744381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59051c-b5be-46e0-8e28-32d96d40377c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:49.746059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59051c-b5be-46e0-8e28-32d96d40377c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:49.757628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 964/1938] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:49.762108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69022615-987c-48ac-9f41-08c04d4baf0a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:49.782153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69022615-987c-48ac-9f41-08c04d4baf0a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:49.787592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef1038-7416-434d-8dea-9ea0742df7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:49.788256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef1038-7416-434d-8dea-9ea0742df7b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:49.788538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef1038-7416-434d-8dea-9ea0742df7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:49.788815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef1038-7416-434d-8dea-9ea0742df7b1 admin admin] PATH_INFO: `/v3/projects/28a85d22f7af46b19cb9603b727ac03c/users/10b18f88a95740f28cb00d41690a015a/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:49.793427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:49.811470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef1038-7416-434d-8dea-9ea0742df7b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28a85d22f7af46b19cb9603b727ac03c, user_id=10b18f88a95740f28cb00d41690a015a, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:49.814206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef1038-7416-434d-8dea-9ea0742df7b1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:49.880442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 975/1939] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:49 2022] PUT /identity/v3/projects/28a85d22f7af46b19cb9603b727ac03c/users/10b18f88a95740f28cb00d41690a015a/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:49.884490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59051c-b5be-46e0-8e28-32d96d40377c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:49.899534 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59051c-b5be-46e0-8e28-32d96d40377c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:49.915332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:49.952797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19357660-d7a6-4a38-92f0-dabe94227481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:49.953442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19357660-d7a6-4a38-92f0-dabe94227481 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:49.953725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19357660-d7a6-4a38-92f0-dabe94227481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:49.954001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19357660-d7a6-4a38-92f0-dabe94227481 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:49.954645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19357660-d7a6-4a38-92f0-dabe94227481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:49.956958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19357660-d7a6-4a38-92f0-dabe94227481 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:49.967038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 965/1940] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:49.971986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef1038-7416-434d-8dea-9ea0742df7b1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:49.982092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef1038-7416-434d-8dea-9ea0742df7b1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:49.986857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924728a4-c921-414b-a5b3-334e360f9a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:49.987589 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924728a4-c921-414b-a5b3-334e360f9a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:49.987869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924728a4-c921-414b-a5b3-334e360f9a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:49.988139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924728a4-c921-414b-a5b3-334e360f9a0a admin admin] PATH_INFO: `/v3/projects/28a85d22f7af46b19cb9603b727ac03c/users/10b18f88a95740f28cb00d41690a015a/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:49.990086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924728a4-c921-414b-a5b3-334e360f9a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=28a85d22f7af46b19cb9603b727ac03c, user_id=10b18f88a95740f28cb00d41690a015a, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:49.992701 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924728a4-c921-414b-a5b3-334e360f9a0a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:50.002839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 976/1941] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:49 2022] PUT /identity/v3/projects/28a85d22f7af46b19cb9603b727ac03c/users/10b18f88a95740f28cb00d41690a015a/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:50.014033 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ea0c2-2aaa-4c67-98d7-d177b55f2e5d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:50.014291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ea0c2-2aaa-4c67-98d7-d177b55f2e5d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:50.014572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ea0c2-2aaa-4c67-98d7-d177b55f2e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:50.042994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a21ea0c2-2aaa-4c67-98d7-d177b55f2e5d None None] MFA Rules not processed for user `10b18f88a95740f28cb00d41690a015a`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:50.044204 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a21ea0c2-2aaa-4c67-98d7-d177b55f2e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:50.127953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 966/1942] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:50.441761 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924728a4-c921-414b-a5b3-334e360f9a0a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:50.443650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-924728a4-c921-414b-a5b3-334e360f9a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:50.503183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924728a4-c921-414b-a5b3-334e360f9a0a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:50.506864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea440bbe-4028-4e88-868c-c024a364f8e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:50.507537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea440bbe-4028-4e88-868c-c024a364f8e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:50.507820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea440bbe-4028-4e88-868c-c024a364f8e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:50.508092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea440bbe-4028-4e88-868c-c024a364f8e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:50.517064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea440bbe-4028-4e88-868c-c024a364f8e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:50.519145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea440bbe-4028-4e88-868c-c024a364f8e2 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:50.528580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 977/1943] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:50.750241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ea0c2-2aaa-4c67-98d7-d177b55f2e5d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:50.751682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a21ea0c2-2aaa-4c67-98d7-d177b55f2e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:50.764416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ea0c2-2aaa-4c67-98d7-d177b55f2e5d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:50.768052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4439d54-5c9c-48cf-8df5-3b4ec43c6e9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:50.768661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4439d54-5c9c-48cf-8df5-3b4ec43c6e9d service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:50.768942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4439d54-5c9c-48cf-8df5-3b4ec43c6e9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:50.769250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4439d54-5c9c-48cf-8df5-3b4ec43c6e9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:50.830615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4439d54-5c9c-48cf-8df5-3b4ec43c6e9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:50.832044 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4439d54-5c9c-48cf-8df5-3b4ec43c6e9d service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:50.848605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 967/1944] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:51.363332 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea440bbe-4028-4e88-868c-c024a364f8e2 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:51.437103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea440bbe-4028-4e88-868c-c024a364f8e2 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:51.442534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:51.443167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:51.443444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:51.443717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] PATH_INFO: `/v3/users/03d8f78baad548c98e16f3ede0f88c0d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:51.444887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d8f78baad548c98e16f3ede0f88c0d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:51.446834 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:51.625183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d8f78baad548c98e16f3ede0f88c0d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:51.632336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d8f78baad548c98e16f3ede0f88c0d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:51.695921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d8f78baad548c98e16f3ede0f88c0d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:51.696646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d8f78baad548c98e16f3ede0f88c0d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:51.707213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:51.709411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d8f78baad548c98e16f3ede0f88c0d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:51.715404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 978/1945] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:51 2022] DELETE /identity/v3/users/03d8f78baad548c98e16f3ede0f88c0d => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:51.719939 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5619b5-cb92-4575-8261-5cf521e23b8d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:51.720166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5619b5-cb92-4575-8261-5cf521e23b8d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:51.720438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5619b5-cb92-4575-8261-5cf521e23b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:51.740429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cd5619b5-cb92-4575-8261-5cf521e23b8d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:51.742423 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd5619b5-cb92-4575-8261-5cf521e23b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:51.782944 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:51.807561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 968/1946] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:00:51.817229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:51.818629 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:51.876093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4da9c-95d0-417d-88f2-5580135095cd admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:51.880094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62b1fa-1e7f-4769-9d70-089a32734d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:51.880814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62b1fa-1e7f-4769-9d70-089a32734d48 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:51.881133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62b1fa-1e7f-4769-9d70-089a32734d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:51.881408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62b1fa-1e7f-4769-9d70-089a32734d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:51.890793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62b1fa-1e7f-4769-9d70-089a32734d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:51.892203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62b1fa-1e7f-4769-9d70-089a32734d48 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:51.901841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 979/1947] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:52.300120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5619b5-cb92-4575-8261-5cf521e23b8d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:52.301981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd5619b5-cb92-4575-8261-5cf521e23b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:52.327695 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5619b5-cb92-4575-8261-5cf521e23b8d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:52.351848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:52.352514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:52.352797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:52.353101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] PATH_INFO: `/v3/projects/609b16aaf0b348c28d722baa78049f12` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:52.354243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=609b16aaf0b348c28d722baa78049f12)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:52.356139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:52.459609 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609b16aaf0b348c28d722baa78049f12', 'request_id': 'req-b72ff7e7-b58c-41a7-ad99-606239787e4c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:52.465785 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609b16aaf0b348c28d722baa78049f12', 'request_id': 'req-b72ff7e7-b58c-41a7-ad99-606239787e4c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:52.472994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609b16aaf0b348c28d722baa78049f12', 'request_id': 'req-b72ff7e7-b58c-41a7-ad99-606239787e4c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:52.524479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] The token cache is being invalidate because project 609b16aaf0b348c28d722baa78049f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:52.524774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:52.525422 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:52.526918 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 969/1948] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:52 2022] DELETE /identity/v3/projects/609b16aaf0b348c28d722baa78049f12 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:52.530947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62b1fa-1e7f-4769-9d70-089a32734d48 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:52.541555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62b1fa-1e7f-4769-9d70-089a32734d48 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:52.596360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3f965-e609-4687-a1e3-1378dde22e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:52.596916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3f965-e609-4687-a1e3-1378dde22e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:52.597219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3f965-e609-4687-a1e3-1378dde22e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:52.597489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3f965-e609-4687-a1e3-1378dde22e8c admin admin] PATH_INFO: `/v3/users/10b18f88a95740f28cb00d41690a015a` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:52.598615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3f965-e609-4687-a1e3-1378dde22e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=10b18f88a95740f28cb00d41690a015a)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:52.600547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3f965-e609-4687-a1e3-1378dde22e8c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:52.963188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-35f3f965-e609-4687-a1e3-1378dde22e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10b18f88a95740f28cb00d41690a015a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:52.970159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-35f3f965-e609-4687-a1e3-1378dde22e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10b18f88a95740f28cb00d41690a015a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:53.090867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-35f3f965-e609-4687-a1e3-1378dde22e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10b18f88a95740f28cb00d41690a015a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:53.091479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-35f3f965-e609-4687-a1e3-1378dde22e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10b18f88a95740f28cb00d41690a015a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:53.104186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-35f3f965-e609-4687-a1e3-1378dde22e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10b18f88a95740f28cb00d41690a015a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:53.109979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 980/1949] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:00:52 2022] DELETE /identity/v3/users/10b18f88a95740f28cb00d41690a015a => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:53.609842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:53.612372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:53.632893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ff7e7-b58c-41a7-ad99-606239787e4c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:53.693931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:53.696797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:53.696797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:53.696797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] PATH_INFO: `/v3/projects/28a85d22f7af46b19cb9603b727ac03c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:53.696797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=28a85d22f7af46b19cb9603b727ac03c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:53.698206 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:53.801277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28a85d22f7af46b19cb9603b727ac03c', 'request_id': 'req-47d1fd10-5072-468c-a027-5fe293f89f3d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:53.807081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28a85d22f7af46b19cb9603b727ac03c', 'request_id': 'req-47d1fd10-5072-468c-a027-5fe293f89f3d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:53.813243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28a85d22f7af46b19cb9603b727ac03c', 'request_id': 'req-47d1fd10-5072-468c-a027-5fe293f89f3d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:53.880434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:53.885200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] The token cache is being invalidate because project 28a85d22f7af46b19cb9603b727ac03c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:00:53.885492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:53.886145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-47d1fd10-5072-468c-a027-5fe293f89f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:00:53.887672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 970/1950] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:00:53 2022] DELETE /identity/v3/projects/28a85d22f7af46b19cb9603b727ac03c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:53.893154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1d290-328c-4e3f-b1de-a8505cf32d0e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:53.893631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1d290-328c-4e3f-b1de-a8505cf32d0e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:53.893869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1d290-328c-4e3f-b1de-a8505cf32d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:53.913171 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-52b1d290-328c-4e3f-b1de-a8505cf32d0e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:53.914382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-52b1d290-328c-4e3f-b1de-a8505cf32d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:53.925307 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:53.978769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 981/1951] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:53.990673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb957d-cf5d-45d6-8b1d-f068931032fd None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:53.990954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb957d-cf5d-45d6-8b1d-f068931032fd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:53.991227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb957d-cf5d-45d6-8b1d-f068931032fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:54.010786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-95cb957d-cf5d-45d6-8b1d-f068931032fd None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:54.011997 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-95cb957d-cf5d-45d6-8b1d-f068931032fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:54.019902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 971/1952] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:00:54.024124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1d290-328c-4e3f-b1de-a8505cf32d0e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:54.034426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1d290-328c-4e3f-b1de-a8505cf32d0e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:54.039275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc8a3b-fbd7-41ac-b92b-967c3be8e1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:54.045571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc8a3b-fbd7-41ac-b92b-967c3be8e1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:54.045571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc8a3b-fbd7-41ac-b92b-967c3be8e1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:54.045571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc8a3b-fbd7-41ac-b92b-967c3be8e1b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:54.045571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc8a3b-fbd7-41ac-b92b-967c3be8e1b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:00:54.045571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc8a3b-fbd7-41ac-b92b-967c3be8e1b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:54.045571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc8a3b-fbd7-41ac-b92b-967c3be8e1b5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:54.057346 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 982/1953] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:00:54.061506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb957d-cf5d-45d6-8b1d-f068931032fd None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:54.071001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb957d-cf5d-45d6-8b1d-f068931032fd None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:54.075688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287fe33-5ae1-4af6-8158-68a5c867c5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:54.076291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287fe33-5ae1-4af6-8158-68a5c867c5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:54.076573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287fe33-5ae1-4af6-8158-68a5c867c5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:54.076845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287fe33-5ae1-4af6-8158-68a5c867c5c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:54.077535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287fe33-5ae1-4af6-8158-68a5c867c5c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:54.079585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287fe33-5ae1-4af6-8158-68a5c867c5c3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:54.190745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 972/1954] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:54 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:54.194971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc8a3b-fbd7-41ac-b92b-967c3be8e1b5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:54.204810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc8a3b-fbd7-41ac-b92b-967c3be8e1b5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:54.259279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f1e3b-c71b-4d78-92e2-86a5758a9e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:54.259878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f1e3b-c71b-4d78-92e2-86a5758a9e04 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:54.260157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f1e3b-c71b-4d78-92e2-86a5758a9e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:54.260429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f1e3b-c71b-4d78-92e2-86a5758a9e04 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:54.261087 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f1e3b-c71b-4d78-92e2-86a5758a9e04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:54.262989 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f1e3b-c71b-4d78-92e2-86a5758a9e04 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:54.356320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 983/1955] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:54 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:00:54.359181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287fe33-5ae1-4af6-8158-68a5c867c5c3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:54.369927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287fe33-5ae1-4af6-8158-68a5c867c5c3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:54.374546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313991c3-11c9-4f92-96e2-f9474d507290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:54.380730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313991c3-11c9-4f92-96e2-f9474d507290 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:54.380730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313991c3-11c9-4f92-96e2-f9474d507290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:54.380730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313991c3-11c9-4f92-96e2-f9474d507290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:54.380730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313991c3-11c9-4f92-96e2-f9474d507290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:54.380730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313991c3-11c9-4f92-96e2-f9474d507290 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:54.387768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 973/1956] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:54.391615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f1e3b-c71b-4d78-92e2-86a5758a9e04 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:54.402670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f1e3b-c71b-4d78-92e2-86a5758a9e04 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:54.407359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab04c5-c968-4cde-9740-1bddc8d9525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:54.412782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab04c5-c968-4cde-9740-1bddc8d9525c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:54.412782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab04c5-c968-4cde-9740-1bddc8d9525c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:54.412782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab04c5-c968-4cde-9740-1bddc8d9525c admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:54.412782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab04c5-c968-4cde-9740-1bddc8d9525c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:54.412782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab04c5-c968-4cde-9740-1bddc8d9525c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:54.421975 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 984/1957] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:54.426133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313991c3-11c9-4f92-96e2-f9474d507290 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:54.440644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313991c3-11c9-4f92-96e2-f9474d507290 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:54.445674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036510c-f2a6-42ea-9aba-4568576be40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:54.446375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036510c-f2a6-42ea-9aba-4568576be40d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:54.446659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036510c-f2a6-42ea-9aba-4568576be40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:54.446952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036510c-f2a6-42ea-9aba-4568576be40d admin admin] PATH_INFO: `/v3/projects/e98a76881e9c41ec9429d3a71a8fd714/users/3065dc56483141108636d4f46c38bff1/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:54.463089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036510c-f2a6-42ea-9aba-4568576be40d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98a76881e9c41ec9429d3a71a8fd714, user_id=3065dc56483141108636d4f46c38bff1, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:54.465768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036510c-f2a6-42ea-9aba-4568576be40d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:54.554923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 974/1958] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:54 2022] PUT /identity/v3/projects/e98a76881e9c41ec9429d3a71a8fd714/users/3065dc56483141108636d4f46c38bff1/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:54.565704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd35f2-eb0e-4f5b-8c7d-23a2f83dd8f7 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:54.565811 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd35f2-eb0e-4f5b-8c7d-23a2f83dd8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:54.566069 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd35f2-eb0e-4f5b-8c7d-23a2f83dd8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:54.586759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-88fd35f2-eb0e-4f5b-8c7d-23a2f83dd8f7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:54.588270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-88fd35f2-eb0e-4f5b-8c7d-23a2f83dd8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:54.674718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 985/1959] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:00:54.685138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036510c-f2a6-42ea-9aba-4568576be40d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:54.687037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6036510c-f2a6-42ea-9aba-4568576be40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:54.768232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036510c-f2a6-42ea-9aba-4568576be40d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:54.781237 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402bf81-77db-4ed4-a543-d3e1474c6b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:54.781871 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402bf81-77db-4ed4-a543-d3e1474c6b3e service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:54.782149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402bf81-77db-4ed4-a543-d3e1474c6b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:54.782421 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402bf81-77db-4ed4-a543-d3e1474c6b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:54.792136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402bf81-77db-4ed4-a543-d3e1474c6b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:54.794963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402bf81-77db-4ed4-a543-d3e1474c6b3e service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:54.803547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 975/1960] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:00:56.075381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80095e47-69ac-47e8-a78a-da4f44de1654 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:56.076282 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80095e47-69ac-47e8-a78a-da4f44de1654 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:56.076556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80095e47-69ac-47e8-a78a-da4f44de1654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:56.109485 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-80095e47-69ac-47e8-a78a-da4f44de1654 None None] MFA Rules not processed for user `3065dc56483141108636d4f46c38bff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:56.110783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-80095e47-69ac-47e8-a78a-da4f44de1654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:56.122529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:56.193584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 986/1961] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:56.198162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402bf81-77db-4ed4-a543-d3e1474c6b3e service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:56.210163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402bf81-77db-4ed4-a543-d3e1474c6b3e service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:56.215937 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a72a5-f45c-43fe-8c27-1e2d070209a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:56.216744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a72a5-f45c-43fe-8c27-1e2d070209a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:56.217331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a72a5-f45c-43fe-8c27-1e2d070209a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:56.217773 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a72a5-f45c-43fe-8c27-1e2d070209a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:56.218577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a72a5-f45c-43fe-8c27-1e2d070209a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:56.220962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a72a5-f45c-43fe-8c27-1e2d070209a9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:56.331627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 976/1962] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:00:56 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:56.343108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80095e47-69ac-47e8-a78a-da4f44de1654 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:56.354574 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80095e47-69ac-47e8-a78a-da4f44de1654 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:56.418248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170062bc-ce0b-4312-b41d-a31ac3dc5093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:56.418838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170062bc-ce0b-4312-b41d-a31ac3dc5093 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:56.419290 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170062bc-ce0b-4312-b41d-a31ac3dc5093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:56.419568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170062bc-ce0b-4312-b41d-a31ac3dc5093 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:56.420570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170062bc-ce0b-4312-b41d-a31ac3dc5093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:56.422552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170062bc-ce0b-4312-b41d-a31ac3dc5093 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:56.546148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 987/1963] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:00:56 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:00:56.547726 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a72a5-f45c-43fe-8c27-1e2d070209a9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:56.558847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a72a5-f45c-43fe-8c27-1e2d070209a9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:56.564158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b36d1-882b-4af8-a24c-41ea6ee536df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:56.569418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b36d1-882b-4af8-a24c-41ea6ee536df admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:56.569418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b36d1-882b-4af8-a24c-41ea6ee536df admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:56.569418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b36d1-882b-4af8-a24c-41ea6ee536df admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:56.569418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b36d1-882b-4af8-a24c-41ea6ee536df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:56.569418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b36d1-882b-4af8-a24c-41ea6ee536df admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:56.581599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 977/1964] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:56.585661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170062bc-ce0b-4312-b41d-a31ac3dc5093 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:56.596547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170062bc-ce0b-4312-b41d-a31ac3dc5093 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:56.601588 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cebb37-65f3-490d-a876-616cdca67f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:56.602281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cebb37-65f3-490d-a876-616cdca67f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:56.602566 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cebb37-65f3-490d-a876-616cdca67f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:56.602839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cebb37-65f3-490d-a876-616cdca67f89 admin admin] PATH_INFO: `/v3/projects/40e254cb662349b8bbf7ccf9ce709c33/users/a05a8b07966a4391982d877d93dcbefe/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:56.620822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cebb37-65f3-490d-a876-616cdca67f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e254cb662349b8bbf7ccf9ce709c33, user_id=a05a8b07966a4391982d877d93dcbefe, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:56.623601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cebb37-65f3-490d-a876-616cdca67f89 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:56.731154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 988/1965] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:56 2022] PUT /identity/v3/projects/40e254cb662349b8bbf7ccf9ce709c33/users/a05a8b07966a4391982d877d93dcbefe/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:56.735160 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b36d1-882b-4af8-a24c-41ea6ee536df admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:56.745674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b36d1-882b-4af8-a24c-41ea6ee536df admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:56.802027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375d35a-0751-46e6-89c0-e91ee183e72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:56.802642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375d35a-0751-46e6-89c0-e91ee183e72f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:56.802927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375d35a-0751-46e6-89c0-e91ee183e72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:56.803202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375d35a-0751-46e6-89c0-e91ee183e72f admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:56.803844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375d35a-0751-46e6-89c0-e91ee183e72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:56.805445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375d35a-0751-46e6-89c0-e91ee183e72f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:56.810380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:56.816241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 978/1966] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:00:56.820486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cebb37-65f3-490d-a876-616cdca67f89 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:56.830910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cebb37-65f3-490d-a876-616cdca67f89 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:56.835508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aab634-84f5-4d8d-b028-490afe8474da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:56.836145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aab634-84f5-4d8d-b028-490afe8474da admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:56.836427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aab634-84f5-4d8d-b028-490afe8474da admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:56.836705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aab634-84f5-4d8d-b028-490afe8474da admin admin] PATH_INFO: `/v3/projects/40e254cb662349b8bbf7ccf9ce709c33/users/a05a8b07966a4391982d877d93dcbefe/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:56.838949 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aab634-84f5-4d8d-b028-490afe8474da admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e254cb662349b8bbf7ccf9ce709c33, user_id=a05a8b07966a4391982d877d93dcbefe, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:56.841605 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aab634-84f5-4d8d-b028-490afe8474da admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:56.852343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 989/1967] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:00:56 2022] PUT /identity/v3/projects/40e254cb662349b8bbf7ccf9ce709c33/users/a05a8b07966a4391982d877d93dcbefe/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:00:58.367996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736cfb0-16cb-4209-8f2d-02597faf96af None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:58.368502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736cfb0-16cb-4209-8f2d-02597faf96af None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:58.368843 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736cfb0-16cb-4209-8f2d-02597faf96af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:58.400596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f736cfb0-16cb-4209-8f2d-02597faf96af None None] MFA Rules not processed for user `a05a8b07966a4391982d877d93dcbefe`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:00:58.401957 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f736cfb0-16cb-4209-8f2d-02597faf96af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:58.476900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 979/1968] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:00:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:00:58.488851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aab634-84f5-4d8d-b028-490afe8474da admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:00:58.491720 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d3aab634-84f5-4d8d-b028-490afe8474da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:00:58.548282 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aab634-84f5-4d8d-b028-490afe8474da admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:00:58.552794 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8443e15-c84b-4fab-8b1b-a3fbf0c497ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:00:58.553432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8443e15-c84b-4fab-8b1b-a3fbf0c497ab service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:00:58.553715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8443e15-c84b-4fab-8b1b-a3fbf0c497ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:00:58.553990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8443e15-c84b-4fab-8b1b-a3fbf0c497ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:00:58.570588 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:00:58.610904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8443e15-c84b-4fab-8b1b-a3fbf0c497ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:00:58.612941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8443e15-c84b-4fab-8b1b-a3fbf0c497ab service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:00:58.624726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 990/1969] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:00:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:00.479094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736cfb0-16cb-4209-8f2d-02597faf96af None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:00.480689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f736cfb0-16cb-4209-8f2d-02597faf96af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:00.493228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736cfb0-16cb-4209-8f2d-02597faf96af None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:00.496949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9e936-c683-43e0-934f-23884baa2337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:00.497656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9e936-c683-43e0-934f-23884baa2337 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:00.497944 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9e936-c683-43e0-934f-23884baa2337 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:00.498222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9e936-c683-43e0-934f-23884baa2337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:00.507841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9e936-c683-43e0-934f-23884baa2337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:00.509324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9e936-c683-43e0-934f-23884baa2337 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:00.520359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 980/1970] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:08.668366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=19 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:10.519455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=40 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:11.681899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8443e15-c84b-4fab-8b1b-a3fbf0c497ab service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:11.683380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e8443e15-c84b-4fab-8b1b-a3fbf0c497ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:11.778641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8443e15-c84b-4fab-8b1b-a3fbf0c497ab service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:11.784112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:11.785124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:11.785498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:11.786934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] PATH_INFO: `/v3/users/2ed9126ca8274ed291fd6c36b374576b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:11.787539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed9126ca8274ed291fd6c36b374576b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:11.792910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:12.023109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed9126ca8274ed291fd6c36b374576b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:12.032168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed9126ca8274ed291fd6c36b374576b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:12.114595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed9126ca8274ed291fd6c36b374576b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:12.115487 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed9126ca8274ed291fd6c36b374576b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:12.128142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed9126ca8274ed291fd6c36b374576b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:12.134625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 991/1971] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:11 2022] DELETE /identity/v3/users/2ed9126ca8274ed291fd6c36b374576b => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:12.139097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7456f8-289b-414f-9acb-b08768c7b301 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:12.139574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7456f8-289b-414f-9acb-b08768c7b301 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:12.139753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7456f8-289b-414f-9acb-b08768c7b301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:12.160628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0d7456f8-289b-414f-9acb-b08768c7b301 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:12.161969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0d7456f8-289b-414f-9acb-b08768c7b301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:12.260491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 981/1972] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:12.275244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:12.276825 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:12.365974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613000f2-cf8b-4302-918a-aedda1c46d2f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:12.371280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a2c3a-7d67-4da7-8c03-80cfa55d55d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:12.371906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a2c3a-7d67-4da7-8c03-80cfa55d55d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:12.372254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a2c3a-7d67-4da7-8c03-80cfa55d55d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:12.372528 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a2c3a-7d67-4da7-8c03-80cfa55d55d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:12.383876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a2c3a-7d67-4da7-8c03-80cfa55d55d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:12.385366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a2c3a-7d67-4da7-8c03-80cfa55d55d0 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:12.397128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 992/1973] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:12.824353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7456f8-289b-414f-9acb-b08768c7b301 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:12.826595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0d7456f8-289b-414f-9acb-b08768c7b301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:12.840930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7456f8-289b-414f-9acb-b08768c7b301 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:12.850840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:12.851529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:12.851907 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:12.852095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] PATH_INFO: `/v3/projects/78ec9e6bf0644cb7a8e3833b3f28fc4e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:12.853482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ec9e6bf0644cb7a8e3833b3f28fc4e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:12.855620 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:12.957097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ec9e6bf0644cb7a8e3833b3f28fc4e', 'request_id': 'req-a0a0602b-3f11-4173-8006-dc91d9bdaa01'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:12.962772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ec9e6bf0644cb7a8e3833b3f28fc4e', 'request_id': 'req-a0a0602b-3f11-4173-8006-dc91d9bdaa01'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:12.969596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ec9e6bf0644cb7a8e3833b3f28fc4e', 'request_id': 'req-a0a0602b-3f11-4173-8006-dc91d9bdaa01'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:13.029606 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] The token cache is being invalidate because project 78ec9e6bf0644cb7a8e3833b3f28fc4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:13.029911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:13.030524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a0a0602b-3f11-4173-8006-dc91d9bdaa01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:13.032223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 982/1974] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:12 2022] DELETE /identity/v3/projects/78ec9e6bf0644cb7a8e3833b3f28fc4e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:13.037609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bb74c-61e9-4d7f-80a2-588c238e5d46 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:13.038384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bb74c-61e9-4d7f-80a2-588c238e5d46 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:13.040208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bb74c-61e9-4d7f-80a2-588c238e5d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:13.060753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-570bb74c-61e9-4d7f-80a2-588c238e5d46 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:13.064862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-570bb74c-61e9-4d7f-80a2-588c238e5d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:13.086936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:13.154574 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 993/1975] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:13.163288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e09c0-ddba-4294-99ac-b89264a07e57 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:13.163834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e09c0-ddba-4294-99ac-b89264a07e57 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:13.163936 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e09c0-ddba-4294-99ac-b89264a07e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:13.191010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7c9e09c0-ddba-4294-99ac-b89264a07e57 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:13.193257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7c9e09c0-ddba-4294-99ac-b89264a07e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:13.225175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 983/1976] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:13.228231 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bb74c-61e9-4d7f-80a2-588c238e5d46 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:13.247552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bb74c-61e9-4d7f-80a2-588c238e5d46 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:13.260799 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1119f1e-a605-4e68-bfde-1d95c1f8b6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:13.267375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1119f1e-a605-4e68-bfde-1d95c1f8b6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:13.267375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1119f1e-a605-4e68-bfde-1d95c1f8b6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:13.267375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1119f1e-a605-4e68-bfde-1d95c1f8b6b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:13.267375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1119f1e-a605-4e68-bfde-1d95c1f8b6b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:13.267375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1119f1e-a605-4e68-bfde-1d95c1f8b6b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:13.267375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1119f1e-a605-4e68-bfde-1d95c1f8b6b9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:13.299148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 994/1977] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:01:13.301073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e09c0-ddba-4294-99ac-b89264a07e57 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:13.331657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e09c0-ddba-4294-99ac-b89264a07e57 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:13.336862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df46ff-0875-4398-b2ff-a93ecf1b1132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:13.337441 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df46ff-0875-4398-b2ff-a93ecf1b1132 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:13.338619 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df46ff-0875-4398-b2ff-a93ecf1b1132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:13.339045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df46ff-0875-4398-b2ff-a93ecf1b1132 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:13.339828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df46ff-0875-4398-b2ff-a93ecf1b1132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:13.342450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df46ff-0875-4398-b2ff-a93ecf1b1132 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:13.489586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 984/1978] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:13 2022] POST /identity/v3/projects => generated 391 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:13.494176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1119f1e-a605-4e68-bfde-1d95c1f8b6b9 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:13.509795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1119f1e-a605-4e68-bfde-1d95c1f8b6b9 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:13.569884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6885635-a1e1-4a31-a514-d9489889b4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:13.570104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6885635-a1e1-4a31-a514-d9489889b4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:13.570315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6885635-a1e1-4a31-a514-d9489889b4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:13.570587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6885635-a1e1-4a31-a514-d9489889b4a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:13.571239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6885635-a1e1-4a31-a514-d9489889b4a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:13.573205 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6885635-a1e1-4a31-a514-d9489889b4a1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:13.692001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 995/1979] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:13 2022] POST /identity/v3/users => generated 296 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:13.695528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df46ff-0875-4398-b2ff-a93ecf1b1132 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:13.717916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df46ff-0875-4398-b2ff-a93ecf1b1132 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:13.723731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0959267-0ef9-4f17-abf0-0f7b16dbba25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:13.724591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0959267-0ef9-4f17-abf0-0f7b16dbba25 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:13.725134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0959267-0ef9-4f17-abf0-0f7b16dbba25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:13.725857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0959267-0ef9-4f17-abf0-0f7b16dbba25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:13.726792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0959267-0ef9-4f17-abf0-0f7b16dbba25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:13.729214 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0959267-0ef9-4f17-abf0-0f7b16dbba25 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:13.760618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 985/1980] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:13.764714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6885635-a1e1-4a31-a514-d9489889b4a1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:13.780055 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6885635-a1e1-4a31-a514-d9489889b4a1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:13.789642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830e092-0a00-4175-83eb-1c213088af1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:13.790261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830e092-0a00-4175-83eb-1c213088af1b admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:13.790540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830e092-0a00-4175-83eb-1c213088af1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:13.791483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830e092-0a00-4175-83eb-1c213088af1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:13.792142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830e092-0a00-4175-83eb-1c213088af1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:13.793822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830e092-0a00-4175-83eb-1c213088af1b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:13.805148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 996/1981] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:13.809660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0959267-0ef9-4f17-abf0-0f7b16dbba25 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:13.820709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0959267-0ef9-4f17-abf0-0f7b16dbba25 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:13.825565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38861b7d-524c-431b-811d-c6373503bc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:13.826246 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38861b7d-524c-431b-811d-c6373503bc67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:13.826745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38861b7d-524c-431b-811d-c6373503bc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:13.827173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38861b7d-524c-431b-811d-c6373503bc67 admin admin] PATH_INFO: `/v3/projects/048d817dd3c54f1389c21eacb4a3a351/users/617fcdadb5474920b1b0f1819000a5b7/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:13.843712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38861b7d-524c-431b-811d-c6373503bc67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=048d817dd3c54f1389c21eacb4a3a351, user_id=617fcdadb5474920b1b0f1819000a5b7, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:13.846567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38861b7d-524c-431b-811d-c6373503bc67 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:13.927493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 986/1982] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:13 2022] PUT /identity/v3/projects/048d817dd3c54f1389c21eacb4a3a351/users/617fcdadb5474920b1b0f1819000a5b7/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:13.942897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534744b-e773-449e-b9fc-80a48f6bc4cf None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:13.947630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534744b-e773-449e-b9fc-80a48f6bc4cf None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:13.947630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534744b-e773-449e-b9fc-80a48f6bc4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:14.019665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-b534744b-e773-449e-b9fc-80a48f6bc4cf None None] MFA Rules not processed for user `617fcdadb5474920b1b0f1819000a5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:14.021777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b534744b-e773-449e-b9fc-80a48f6bc4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:14.119787 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 997/1983] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:14.132347 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38861b7d-524c-431b-811d-c6373503bc67 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:14.134161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-38861b7d-524c-431b-811d-c6373503bc67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:14.155681 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534744b-e773-449e-b9fc-80a48f6bc4cf None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:14.157571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b534744b-e773-449e-b9fc-80a48f6bc4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:14.166021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:14.191750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:14.217415 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38861b7d-524c-431b-811d-c6373503bc67 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:14.222220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f50f7-8fe0-4d7e-a743-9998e5a1935a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:14.223227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f50f7-8fe0-4d7e-a743-9998e5a1935a service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:14.223808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f50f7-8fe0-4d7e-a743-9998e5a1935a service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:14.224254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f50f7-8fe0-4d7e-a743-9998e5a1935a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:14.238855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f50f7-8fe0-4d7e-a743-9998e5a1935a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:14.245760 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f50f7-8fe0-4d7e-a743-9998e5a1935a service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:14.252682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534744b-e773-449e-b9fc-80a48f6bc4cf None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:14.257868 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:14.261235 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:14.261359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 987/1984] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:14.261469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:14.261469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] PATH_INFO: `/v3/users/99d2a1ac104a49cbb52da25742f17ec9` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:14.269231 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d2a1ac104a49cbb52da25742f17ec9)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:14.271669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:14.511122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d2a1ac104a49cbb52da25742f17ec9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:14.521150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d2a1ac104a49cbb52da25742f17ec9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:14.591430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d2a1ac104a49cbb52da25742f17ec9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:14.592085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d2a1ac104a49cbb52da25742f17ec9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:14.606834 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d2a1ac104a49cbb52da25742f17ec9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:14.617464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 998/1985] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:14 2022] DELETE /identity/v3/users/99d2a1ac104a49cbb52da25742f17ec9 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:14.622352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e23daf-6f34-465c-b220-b17b8cc1d000 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:14.622659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e23daf-6f34-465c-b220-b17b8cc1d000 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:14.623093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e23daf-6f34-465c-b220-b17b8cc1d000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:14.659234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-51e23daf-6f34-465c-b220-b17b8cc1d000 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:14.660734 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-51e23daf-6f34-465c-b220-b17b8cc1d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:14.760703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 988/1986] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:14.772758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:14.774418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:14.859435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba9fc9-1720-4f22-81e2-878c91c5ab2f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:14.867284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697efaa-90b2-468d-8245-11c08870b470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:14.867910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697efaa-90b2-468d-8245-11c08870b470 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:14.868190 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697efaa-90b2-468d-8245-11c08870b470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:14.868510 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697efaa-90b2-468d-8245-11c08870b470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:14.880522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697efaa-90b2-468d-8245-11c08870b470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:14.881992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697efaa-90b2-468d-8245-11c08870b470 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:14.894268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 999/1987] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:14.894770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e23daf-6f34-465c-b220-b17b8cc1d000 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:14.896475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-51e23daf-6f34-465c-b220-b17b8cc1d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:14.911594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e23daf-6f34-465c-b220-b17b8cc1d000 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:14.919173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8af61-e7b7-4029-8ab0-0150e8aa8949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:14.919341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8af61-e7b7-4029-8ab0-0150e8aa8949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:14.919562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8af61-e7b7-4029-8ab0-0150e8aa8949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:14.919993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8af61-e7b7-4029-8ab0-0150e8aa8949 admin admin] PATH_INFO: `/v3/users/08c32b0e61c244d6b49ac467b6cde77e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:14.921234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8af61-e7b7-4029-8ab0-0150e8aa8949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c32b0e61c244d6b49ac467b6cde77e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:14.923964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8af61-e7b7-4029-8ab0-0150e8aa8949 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:15.023761 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697efaa-90b2-468d-8245-11c08870b470 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:15.025338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c697efaa-90b2-468d-8245-11c08870b470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:15.042257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697efaa-90b2-468d-8245-11c08870b470 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:15.047114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:15.047857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:15.048934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:15.049405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] PATH_INFO: `/v3/users/bf9353bc130d45a99af9386ff9dd9a2f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:15.050554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf9353bc130d45a99af9386ff9dd9a2f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:15.052448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:15.135399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-39f8af61-e7b7-4029-8ab0-0150e8aa8949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c32b0e61c244d6b49ac467b6cde77e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.136125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-39f8af61-e7b7-4029-8ab0-0150e8aa8949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c32b0e61c244d6b49ac467b6cde77e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.136749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-39f8af61-e7b7-4029-8ab0-0150e8aa8949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c32b0e61c244d6b49ac467b6cde77e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.228294 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-39f8af61-e7b7-4029-8ab0-0150e8aa8949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c32b0e61c244d6b49ac467b6cde77e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.253405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-39f8af61-e7b7-4029-8ab0-0150e8aa8949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c32b0e61c244d6b49ac467b6cde77e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.264108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:15.274175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 989/1988] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:14 2022] DELETE /identity/v3/users/08c32b0e61c244d6b49ac467b6cde77e => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:15.275316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf9353bc130d45a99af9386ff9dd9a2f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.278783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7812ae5-43b7-4e7a-a205-98b305d2510c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:15.279311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7812ae5-43b7-4e7a-a205-98b305d2510c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:15.279595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7812ae5-43b7-4e7a-a205-98b305d2510c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:15.284441 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf9353bc130d45a99af9386ff9dd9a2f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.300292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c7812ae5-43b7-4e7a-a205-98b305d2510c None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:15.301858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c7812ae5-43b7-4e7a-a205-98b305d2510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:15.332373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:15.370450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 990/1989] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:15.374743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7812ae5-43b7-4e7a-a205-98b305d2510c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:15.376090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c7812ae5-43b7-4e7a-a205-98b305d2510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:15.388739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7812ae5-43b7-4e7a-a205-98b305d2510c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:15.394665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:15.395302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:15.395586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:15.395861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] PATH_INFO: `/v3/projects/4ed0ea5ffa3e453689ce34ea857cd47a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:15.396988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ed0ea5ffa3e453689ce34ea857cd47a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:15.398915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:15.427836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf9353bc130d45a99af9386ff9dd9a2f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.428488 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf9353bc130d45a99af9386ff9dd9a2f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.441368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf9353bc130d45a99af9386ff9dd9a2f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.447461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1000/1990] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:15 2022] DELETE /identity/v3/users/bf9353bc130d45a99af9386ff9dd9a2f => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:15.448238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:15.449586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:15.521081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e302db2-5b52-45dc-96b5-d304b1e49ecf admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:15.524843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad427b0c-5c66-4575-957e-29c4cc89aa2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:15.525494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad427b0c-5c66-4575-957e-29c4cc89aa2f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:15.525769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad427b0c-5c66-4575-957e-29c4cc89aa2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:15.526036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad427b0c-5c66-4575-957e-29c4cc89aa2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:15.527790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ad427b0c-5c66-4575-957e-29c4cc89aa2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:15.541907 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed0ea5ffa3e453689ce34ea857cd47a', 'request_id': 'req-533f0f4a-44f6-4e27-8707-5060f9083c03'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.554338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed0ea5ffa3e453689ce34ea857cd47a', 'request_id': 'req-533f0f4a-44f6-4e27-8707-5060f9083c03'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.565608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed0ea5ffa3e453689ce34ea857cd47a', 'request_id': 'req-533f0f4a-44f6-4e27-8707-5060f9083c03'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.609822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad427b0c-5c66-4575-957e-29c4cc89aa2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:15.612015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad427b0c-5c66-4575-957e-29c4cc89aa2f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:15.622637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1001/1991] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:15.639426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] The token cache is being invalidate because project 4ed0ea5ffa3e453689ce34ea857cd47a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:15.639764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.640542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.642388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 991/1992] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:15 2022] DELETE /identity/v3/projects/4ed0ea5ffa3e453689ce34ea857cd47a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:15.646294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad427b0c-5c66-4575-957e-29c4cc89aa2f service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:15.647700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ad427b0c-5c66-4575-957e-29c4cc89aa2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:15.661372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad427b0c-5c66-4575-957e-29c4cc89aa2f service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:15.731109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:15.731716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:15.731997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:15.732274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] PATH_INFO: `/v3/users/993624e46def411db786605ff9309f99` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:15.733456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=993624e46def411db786605ff9309f99)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:15.735385 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:15.761938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:15.763385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:15.784653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f0f4a-44f6-4e27-8707-5060f9083c03 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:15.793400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:15.794215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:15.794971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:15.794971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] PATH_INFO: `/v3/projects/9f6ae51a6843404299c66e97cec51fec` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:15.796466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f6ae51a6843404299c66e97cec51fec)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:15.801284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:15.936933 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f6ae51a6843404299c66e97cec51fec', 'request_id': 'req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.939836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993624e46def411db786605ff9309f99'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.944193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f6ae51a6843404299c66e97cec51fec', 'request_id': 'req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.950435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993624e46def411db786605ff9309f99'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:15.954147 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f6ae51a6843404299c66e97cec51fec', 'request_id': 'req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.068270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993624e46def411db786605ff9309f99'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.068976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993624e46def411db786605ff9309f99'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.082379 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993624e46def411db786605ff9309f99'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.088525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1002/1993] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:15 2022] DELETE /identity/v3/users/993624e46def411db786605ff9309f99 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:16.151738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:16.153250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:16.178097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] The token cache is being invalidate because project 9f6ae51a6843404299c66e97cec51fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:16.178395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.182205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-17cf45d3-bda4-408b-8a3f-c2d6ca88e910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.187058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 992/1994] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:15 2022] DELETE /identity/v3/projects/9f6ae51a6843404299c66e97cec51fec => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:16.201279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5a375-3b80-42ac-b094-cce373edb9d5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.202116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5a375-3b80-42ac-b094-cce373edb9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.203234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5a375-3b80-42ac-b094-cce373edb9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.243189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-36e5a375-3b80-42ac-b094-cce373edb9d5 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:16.245342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-36e5a375-3b80-42ac-b094-cce373edb9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:16.258164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea018de-c5c2-406d-8497-f3cdfec343a6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:16.262944 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:16.263585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.263864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.264293 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] PATH_INFO: `/v3/projects/5c553633f5654aed9c03c9b716083603` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.265474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c553633f5654aed9c03c9b716083603)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:16.266654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 993/1995] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:16.267967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:16.268741 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426ccf9-0fa7-4db5-9dad-08d736780195 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.269247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426ccf9-0fa7-4db5-9dad-08d736780195 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.269530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426ccf9-0fa7-4db5-9dad-08d736780195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.293108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a426ccf9-0fa7-4db5-9dad-08d736780195 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:16.294420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a426ccf9-0fa7-4db5-9dad-08d736780195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:16.303748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 994/1996] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:16.308038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426ccf9-0fa7-4db5-9dad-08d736780195 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:16.317522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426ccf9-0fa7-4db5-9dad-08d736780195 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:16.322343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014858f-3260-491b-b5f2-b42c84d21efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:16.322945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014858f-3260-491b-b5f2-b42c84d21efd admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.323250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014858f-3260-491b-b5f2-b42c84d21efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.323518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014858f-3260-491b-b5f2-b42c84d21efd admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.324172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014858f-3260-491b-b5f2-b42c84d21efd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:16.324462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014858f-3260-491b-b5f2-b42c84d21efd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:16.326100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014858f-3260-491b-b5f2-b42c84d21efd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:16.338917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 995/1997] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:01:16.343345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014858f-3260-491b-b5f2-b42c84d21efd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:16.359696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014858f-3260-491b-b5f2-b42c84d21efd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:16.361559 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df6ba0-0d2f-4478-a09b-208c543ea367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:16.362179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df6ba0-0d2f-4478-a09b-208c543ea367 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.362458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df6ba0-0d2f-4478-a09b-208c543ea367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.362729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df6ba0-0d2f-4478-a09b-208c543ea367 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.363361 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df6ba0-0d2f-4478-a09b-208c543ea367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:16.365314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df6ba0-0d2f-4478-a09b-208c543ea367 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:16.417883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c553633f5654aed9c03c9b716083603', 'request_id': 'req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.424013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c553633f5654aed9c03c9b716083603', 'request_id': 'req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.431583 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c553633f5654aed9c03c9b716083603', 'request_id': 'req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.498596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 996/1998] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:16 2022] POST /identity/v3/projects => generated 387 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:16.503284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df6ba0-0d2f-4478-a09b-208c543ea367 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:16.524791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] The token cache is being invalidate because project 5c553633f5654aed9c03c9b716083603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:16.525109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.525754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a15ca3b6-c2b6-4f12-ba7e-3c8d675b10f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.529638 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1003/1999] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:16 2022] DELETE /identity/v3/projects/5c553633f5654aed9c03c9b716083603 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:16.532921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffc881-ee29-4afe-8812-256b8a4b84ef None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.533236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffc881-ee29-4afe-8812-256b8a4b84ef None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.533516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffc881-ee29-4afe-8812-256b8a4b84ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.552314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-39ffc881-ee29-4afe-8812-256b8a4b84ef None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:16.553617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-39ffc881-ee29-4afe-8812-256b8a4b84ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:16.565300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df6ba0-0d2f-4478-a09b-208c543ea367 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:16.569954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1b631-ba4d-475f-87c2-590d9872c000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:16.570559 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1b631-ba4d-475f-87c2-590d9872c000 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.570841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1b631-ba4d-475f-87c2-590d9872c000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.571118 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1b631-ba4d-475f-87c2-590d9872c000 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.571744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1b631-ba4d-475f-87c2-590d9872c000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:16.573702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1b631-ba4d-475f-87c2-590d9872c000 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:16.636196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1004/2000] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:16.637161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffc881-ee29-4afe-8812-256b8a4b84ef None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:16.638495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-39ffc881-ee29-4afe-8812-256b8a4b84ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:16.651692 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffc881-ee29-4afe-8812-256b8a4b84ef None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:16.656479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:16.657135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.657453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.657819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] PATH_INFO: `/v3/projects/36d2812e93ea4d9d82d2540d3ec4ffa7` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.659107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d2812e93ea4d9d82d2540d3ec4ffa7)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:16.661033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:16.735577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 997/2001] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:16 2022] POST /identity/v3/users => generated 294 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:16.735577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39121ad-69e5-40bb-b1a2-3f41c4b46506 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.735577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39121ad-69e5-40bb-b1a2-3f41c4b46506 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.735577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39121ad-69e5-40bb-b1a2-3f41c4b46506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.756414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b39121ad-69e5-40bb-b1a2-3f41c4b46506 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:16.757678 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b39121ad-69e5-40bb-b1a2-3f41c4b46506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:16.772913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d2812e93ea4d9d82d2540d3ec4ffa7', 'request_id': 'req-29d23a33-7146-410c-8501-2ca65dd2e57c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.778475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d2812e93ea4d9d82d2540d3ec4ffa7', 'request_id': 'req-29d23a33-7146-410c-8501-2ca65dd2e57c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.785098 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d2812e93ea4d9d82d2540d3ec4ffa7', 'request_id': 'req-29d23a33-7146-410c-8501-2ca65dd2e57c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.820284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 998/2002] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:16.821170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39121ad-69e5-40bb-b1a2-3f41c4b46506 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:16.822469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b39121ad-69e5-40bb-b1a2-3f41c4b46506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:16.835235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39121ad-69e5-40bb-b1a2-3f41c4b46506 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:16.839570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] The token cache is being invalidate because project 36d2812e93ea4d9d82d2540d3ec4ffa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:16.839862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.841111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa0e40-4a13-4726-9c0e-069cda528903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:16.841234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:16.841323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa0e40-4a13-4726-9c0e-069cda528903 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.841424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa0e40-4a13-4726-9c0e-069cda528903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.841682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa0e40-4a13-4726-9c0e-069cda528903 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.842106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1005/2003] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:16 2022] DELETE /identity/v3/projects/36d2812e93ea4d9d82d2540d3ec4ffa7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:16.842323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa0e40-4a13-4726-9c0e-069cda528903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:16.842862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:16.844051 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa0e40-4a13-4726-9c0e-069cda528903 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:16.844157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:16.857751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 999/2004] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:16.857751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce28ec-e24c-4c43-b4ee-ef127d3a90f2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.857751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce28ec-e24c-4c43-b4ee-ef127d3a90f2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.857751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce28ec-e24c-4c43-b4ee-ef127d3a90f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.858121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d23a33-7146-410c-8501-2ca65dd2e57c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:16.862392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ce559-2a5a-4f1a-8ab6-7366c6ee1adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:16.862987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ce559-2a5a-4f1a-8ab6-7366c6ee1adf admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.863268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ce559-2a5a-4f1a-8ab6-7366c6ee1adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.863537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ce559-2a5a-4f1a-8ab6-7366c6ee1adf admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.864178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ce559-2a5a-4f1a-8ab6-7366c6ee1adf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:16.868854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ce559-2a5a-4f1a-8ab6-7366c6ee1adf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:16.868854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ce559-2a5a-4f1a-8ab6-7366c6ee1adf admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:16.878368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a9ce28ec-e24c-4c43-b4ee-ef127d3a90f2 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:16.878606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1006/2005] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:01:16.879371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ce559-2a5a-4f1a-8ab6-7366c6ee1adf admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:16.879572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a9ce28ec-e24c-4c43-b4ee-ef127d3a90f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:16.881078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b59ce559-2a5a-4f1a-8ab6-7366c6ee1adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:16.888209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1000/2006] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:16.888930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce28ec-e24c-4c43-b4ee-ef127d3a90f2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:16.893940 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ce559-2a5a-4f1a-8ab6-7366c6ee1adf admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:16.899057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce28ec-e24c-4c43-b4ee-ef127d3a90f2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:16.899299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51959737-6064-4b7e-8569-81217fde2b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:16.899913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51959737-6064-4b7e-8569-81217fde2b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.900181 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51959737-6064-4b7e-8569-81217fde2b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.900451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51959737-6064-4b7e-8569-81217fde2b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.901179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51959737-6064-4b7e-8569-81217fde2b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:16.902772 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51959737-6064-4b7e-8569-81217fde2b6d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:16.904254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd500c-5140-4f82-b52f-9530256ac4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:16.904920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd500c-5140-4f82-b52f-9530256ac4c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.905232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd500c-5140-4f82-b52f-9530256ac4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.905688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd500c-5140-4f82-b52f-9530256ac4c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.906392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd500c-5140-4f82-b52f-9530256ac4c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:16.908433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd500c-5140-4f82-b52f-9530256ac4c3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:16.915855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1007/2007] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:16.920622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750769ad-8247-45b0-b2cc-624ad44de432 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.920622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750769ad-8247-45b0-b2cc-624ad44de432 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.920622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750769ad-8247-45b0-b2cc-624ad44de432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.948204 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-750769ad-8247-45b0-b2cc-624ad44de432 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:16.949440 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-750769ad-8247-45b0-b2cc-624ad44de432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:16.957677 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1008/2008] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:16.960930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750769ad-8247-45b0-b2cc-624ad44de432 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:16.968338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750769ad-8247-45b0-b2cc-624ad44de432 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:16.973202 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23ac9e-fbbc-47b6-86bc-42f5b2f2a322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:16.973847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23ac9e-fbbc-47b6-86bc-42f5b2f2a322 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:16.974124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23ac9e-fbbc-47b6-86bc-42f5b2f2a322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:16.974395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23ac9e-fbbc-47b6-86bc-42f5b2f2a322 admin admin] PATH_INFO: `/v3/projects/a8e4d096ae8d4610a665f2c3f197d9b0/users/6113630b377f40eda2f8994ae69633ab/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:16.991157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23ac9e-fbbc-47b6-86bc-42f5b2f2a322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e4d096ae8d4610a665f2c3f197d9b0, user_id=6113630b377f40eda2f8994ae69633ab, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:16.993892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23ac9e-fbbc-47b6-86bc-42f5b2f2a322 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:16.998443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1001/2009] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:16 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:16.998443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd500c-5140-4f82-b52f-9530256ac4c3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.058431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd500c-5140-4f82-b52f-9530256ac4c3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.063157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b9223-a826-40bc-8705-c6cea2c53309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:17.063884 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b9223-a826-40bc-8705-c6cea2c53309 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.064168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b9223-a826-40bc-8705-c6cea2c53309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.064445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b9223-a826-40bc-8705-c6cea2c53309 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.065137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b9223-a826-40bc-8705-c6cea2c53309 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:17.065442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b9223-a826-40bc-8705-c6cea2c53309 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:17.067060 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b9223-a826-40bc-8705-c6cea2c53309 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:17.076682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:17.080236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1002/2010] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:01:17.081927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b9223-a826-40bc-8705-c6cea2c53309 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.091925 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1009/2011] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:16 2022] PUT /identity/v3/projects/a8e4d096ae8d4610a665f2c3f197d9b0/users/6113630b377f40eda2f8994ae69633ab/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:17.091925 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23ac9e-fbbc-47b6-86bc-42f5b2f2a322 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.100253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b9223-a826-40bc-8705-c6cea2c53309 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.189632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23ac9e-fbbc-47b6-86bc-42f5b2f2a322 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.196840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834325af-f519-4831-8c5d-e798e5c049ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:17.197738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834325af-f519-4831-8c5d-e798e5c049ea admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.198599 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834325af-f519-4831-8c5d-e798e5c049ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.199094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834325af-f519-4831-8c5d-e798e5c049ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.199950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834325af-f519-4831-8c5d-e798e5c049ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:17.203226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834325af-f519-4831-8c5d-e798e5c049ea admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:17.217139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:17.225554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409fe9d-bb84-4844-a0f6-6f3a9f38b87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:17.226654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409fe9d-bb84-4844-a0f6-6f3a9f38b87f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.227109 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409fe9d-bb84-4844-a0f6-6f3a9f38b87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.227583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409fe9d-bb84-4844-a0f6-6f3a9f38b87f admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.228670 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409fe9d-bb84-4844-a0f6-6f3a9f38b87f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:17.232536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409fe9d-bb84-4844-a0f6-6f3a9f38b87f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:17.334490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1003/2012] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:17 2022] POST /identity/v3/users => generated 288 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:17.335541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1010/2013] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:17 2022] POST /identity/v3/projects => generated 387 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:17.336373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8430dc-8e0a-437f-abae-1d584358de4d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.336672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8430dc-8e0a-437f-abae-1d584358de4d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.336952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8430dc-8e0a-437f-abae-1d584358de4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.342352 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834325af-f519-4831-8c5d-e798e5c049ea admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.357416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-af8430dc-8e0a-437f-abae-1d584358de4d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:17.359391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-af8430dc-8e0a-437f-abae-1d584358de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:17.415226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834325af-f519-4831-8c5d-e798e5c049ea admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.420244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd7dcb-6a73-4392-bf78-5c7a6f1edb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:17.420864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd7dcb-6a73-4392-bf78-5c7a6f1edb32 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.421243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd7dcb-6a73-4392-bf78-5c7a6f1edb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.421456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd7dcb-6a73-4392-bf78-5c7a6f1edb32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.432604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd7dcb-6a73-4392-bf78-5c7a6f1edb32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:17.432604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd7dcb-6a73-4392-bf78-5c7a6f1edb32 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:17.437257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1011/2014] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:17.438426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd7dcb-6a73-4392-bf78-5c7a6f1edb32 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.451709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1004/2015] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:17.452013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd7dcb-6a73-4392-bf78-5c7a6f1edb32 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.452586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8430dc-8e0a-437f-abae-1d584358de4d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.458612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84f5ec-0f25-4358-8ee0-7064691ebdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:17.459511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84f5ec-0f25-4358-8ee0-7064691ebdcc admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.459801 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84f5ec-0f25-4358-8ee0-7064691ebdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.460159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84f5ec-0f25-4358-8ee0-7064691ebdcc admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.461697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84f5ec-0f25-4358-8ee0-7064691ebdcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:17.464181 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84f5ec-0f25-4358-8ee0-7064691ebdcc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:17.464999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8430dc-8e0a-437f-abae-1d584358de4d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.471084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d06ee-dc20-4fff-a568-170b8b4205d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:17.471781 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d06ee-dc20-4fff-a568-170b8b4205d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.472286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d06ee-dc20-4fff-a568-170b8b4205d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.472568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d06ee-dc20-4fff-a568-170b8b4205d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.473336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d06ee-dc20-4fff-a568-170b8b4205d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:17.493178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d06ee-dc20-4fff-a568-170b8b4205d6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:17.493178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1005/2016] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:17.493178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d06ee-dc20-4fff-a568-170b8b4205d6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.493178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1d5d06ee-dc20-4fff-a568-170b8b4205d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:17.556809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d06ee-dc20-4fff-a568-170b8b4205d6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.560582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71530f4-b930-4cf4-a172-61ea4ba4342f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:17.561222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71530f4-b930-4cf4-a172-61ea4ba4342f service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.561497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71530f4-b930-4cf4-a172-61ea4ba4342f service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.561767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71530f4-b930-4cf4-a172-61ea4ba4342f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.571256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71530f4-b930-4cf4-a172-61ea4ba4342f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:17.572661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71530f4-b930-4cf4-a172-61ea4ba4342f service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:17.582664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1006/2017] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:17.583440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71530f4-b930-4cf4-a172-61ea4ba4342f service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.592644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71530f4-b930-4cf4-a172-61ea4ba4342f service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.598234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e10eb-fc8d-4e24-a3b3-0a7f0eda7f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:17.598880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e10eb-fc8d-4e24-a3b3-0a7f0eda7f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.599154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e10eb-fc8d-4e24-a3b3-0a7f0eda7f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.599470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e10eb-fc8d-4e24-a3b3-0a7f0eda7f4e admin admin] PATH_INFO: `/v3/projects/399c4a938be94e549b963d3b4461a788/users/3119f622d618465e9c8cb3a35aa2e065/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.620389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e10eb-fc8d-4e24-a3b3-0a7f0eda7f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=399c4a938be94e549b963d3b4461a788, user_id=3119f622d618465e9c8cb3a35aa2e065, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:17.620389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e10eb-fc8d-4e24-a3b3-0a7f0eda7f4e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:17.656816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1012/2018] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:17 2022] POST /identity/v3/users => generated 294 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:17.661388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84f5ec-0f25-4358-8ee0-7064691ebdcc admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.670676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84f5ec-0f25-4358-8ee0-7064691ebdcc admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.675299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814699b-d9e4-461a-89b3-6ee36642713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:17.675871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814699b-d9e4-461a-89b3-6ee36642713e admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.676146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814699b-d9e4-461a-89b3-6ee36642713e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.676415 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814699b-d9e4-461a-89b3-6ee36642713e admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.677071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814699b-d9e4-461a-89b3-6ee36642713e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:17.680706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814699b-d9e4-461a-89b3-6ee36642713e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:17.693283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1013/2019] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:17.697155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814699b-d9e4-461a-89b3-6ee36642713e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.707359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814699b-d9e4-461a-89b3-6ee36642713e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.712269 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3a713-5f63-4665-9e4d-5372f725aabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:17.712830 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a713-5f63-4665-9e4d-5372f725aabd admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.713389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a713-5f63-4665-9e4d-5372f725aabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.713657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a713-5f63-4665-9e4d-5372f725aabd admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.714294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3a713-5f63-4665-9e4d-5372f725aabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:17.715847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3a713-5f63-4665-9e4d-5372f725aabd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:17.726887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1014/2020] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:17.731061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3a713-5f63-4665-9e4d-5372f725aabd admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.740977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3a713-5f63-4665-9e4d-5372f725aabd admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.745974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22839ff-5df8-43dc-aa0c-5cd98807cce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:17.746638 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22839ff-5df8-43dc-aa0c-5cd98807cce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.746922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22839ff-5df8-43dc-aa0c-5cd98807cce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.747227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22839ff-5df8-43dc-aa0c-5cd98807cce4 admin admin] PATH_INFO: `/v3/projects/a9869a793456413fad427ac04df6b8bf/users/99bb5e17e42b4f8eb1790a603b64bd1d/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.751558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1007/2021] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:17 2022] PUT /identity/v3/projects/399c4a938be94e549b963d3b4461a788/users/3119f622d618465e9c8cb3a35aa2e065/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:17.762979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bb479-6ab5-486f-8299-c9161291ae18 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.763487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bb479-6ab5-486f-8299-c9161291ae18 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.764046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22839ff-5df8-43dc-aa0c-5cd98807cce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9869a793456413fad427ac04df6b8bf, user_id=99bb5e17e42b4f8eb1790a603b64bd1d, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:17.764281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bb479-6ab5-486f-8299-c9161291ae18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.769240 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22839ff-5df8-43dc-aa0c-5cd98807cce4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:17.805825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cb3bb479-6ab5-486f-8299-c9161291ae18 None None] MFA Rules not processed for user `3119f622d618465e9c8cb3a35aa2e065`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:17.807001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cb3bb479-6ab5-486f-8299-c9161291ae18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:17.856758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1015/2022] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:17 2022] PUT /identity/v3/projects/a9869a793456413fad427ac04df6b8bf/users/99bb5e17e42b4f8eb1790a603b64bd1d/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:17.872686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1008/2023] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:17.878828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bb479-6ab5-486f-8299-c9161291ae18 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:17.879685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf93457-e3dd-47dc-85ff-1dc26a913b22 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:17.879828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf93457-e3dd-47dc-85ff-1dc26a913b22 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:17.880036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf93457-e3dd-47dc-85ff-1dc26a913b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:17.912499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-cbf93457-e3dd-47dc-85ff-1dc26a913b22 None None] MFA Rules not processed for user `99bb5e17e42b4f8eb1790a603b64bd1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:17.913820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cbf93457-e3dd-47dc-85ff-1dc26a913b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:17.955586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bb479-6ab5-486f-8299-c9161291ae18 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:17.985152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1016/2024] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:17.989894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf93457-e3dd-47dc-85ff-1dc26a913b22 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.019640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e9ee8-cf34-4b0f-9b51-e68834f3038f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.020261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e9ee8-cf34-4b0f-9b51-e68834f3038f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.020552 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e9ee8-cf34-4b0f-9b51-e68834f3038f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.020871 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e9ee8-cf34-4b0f-9b51-e68834f3038f admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.021535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e9ee8-cf34-4b0f-9b51-e68834f3038f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.023429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e9ee8-cf34-4b0f-9b51-e68834f3038f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.052252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf93457-e3dd-47dc-85ff-1dc26a913b22 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:18.057211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89b556-5c22-4b5a-8989-322f684d68fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.073224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89b556-5c22-4b5a-8989-322f684d68fd admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.073224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89b556-5c22-4b5a-8989-322f684d68fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.073224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89b556-5c22-4b5a-8989-322f684d68fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.073224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89b556-5c22-4b5a-8989-322f684d68fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.073224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89b556-5c22-4b5a-8989-322f684d68fd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.145363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1009/2025] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:17 2022] POST /identity/v3/projects => generated 373 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:18.147384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e9ee8-cf34-4b0f-9b51-e68834f3038f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.197685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:18.214613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1017/2026] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:17 2022] POST /identity/v3/projects => generated 387 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:18.218906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89b556-5c22-4b5a-8989-322f684d68fd admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.233249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e9ee8-cf34-4b0f-9b51-e68834f3038f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:18.297636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a5b4e-155a-40da-af10-0aa9bc0c7577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.298664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a5b4e-155a-40da-af10-0aa9bc0c7577 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.299163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a5b4e-155a-40da-af10-0aa9bc0c7577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.299608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a5b4e-155a-40da-af10-0aa9bc0c7577 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.300613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a5b4e-155a-40da-af10-0aa9bc0c7577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.300773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89b556-5c22-4b5a-8989-322f684d68fd admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:18.302980 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a5b4e-155a-40da-af10-0aa9bc0c7577 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.305637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acab9f-9e39-477e-8791-e1f3808ac635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.306245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acab9f-9e39-477e-8791-e1f3808ac635 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.306527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acab9f-9e39-477e-8791-e1f3808ac635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.306804 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acab9f-9e39-477e-8791-e1f3808ac635 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.307439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acab9f-9e39-477e-8791-e1f3808ac635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.309431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0acab9f-9e39-477e-8791-e1f3808ac635 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.320120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:18.410487 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1018/2027] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:18 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:18.412266 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acab9f-9e39-477e-8791-e1f3808ac635 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.413361 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1010/2028] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:18 2022] POST /identity/v3/users => generated 287 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:18.418159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a5b4e-155a-40da-af10-0aa9bc0c7577 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.421502 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acab9f-9e39-477e-8791-e1f3808ac635 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:18.429333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a5b4e-155a-40da-af10-0aa9bc0c7577 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:18.434134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec569bd5-43fb-4eed-a8e5-960c141eeaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.434753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec569bd5-43fb-4eed-a8e5-960c141eeaa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.435039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec569bd5-43fb-4eed-a8e5-960c141eeaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.435314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec569bd5-43fb-4eed-a8e5-960c141eeaa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.439565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec569bd5-43fb-4eed-a8e5-960c141eeaa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.439565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec569bd5-43fb-4eed-a8e5-960c141eeaa4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.439837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30217463-58bd-45c0-888a-ad3bd1a351bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.439928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30217463-58bd-45c0-888a-ad3bd1a351bc admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.440252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30217463-58bd-45c0-888a-ad3bd1a351bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.440416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30217463-58bd-45c0-888a-ad3bd1a351bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.441247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30217463-58bd-45c0-888a-ad3bd1a351bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.442918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30217463-58bd-45c0-888a-ad3bd1a351bc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.449264 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1011/2029] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:18.453388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec569bd5-43fb-4eed-a8e5-960c141eeaa4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.461263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1019/2030] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:18.462306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30217463-58bd-45c0-888a-ad3bd1a351bc admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.466944 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec569bd5-43fb-4eed-a8e5-960c141eeaa4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:18.473878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30217463-58bd-45c0-888a-ad3bd1a351bc admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:18.476488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50790a1f-c4e4-43be-b851-2bfba6229b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.477323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50790a1f-c4e4-43be-b851-2bfba6229b3e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.477611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50790a1f-c4e4-43be-b851-2bfba6229b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.477887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50790a1f-c4e4-43be-b851-2bfba6229b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.478541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50790a1f-c4e4-43be-b851-2bfba6229b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.479347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76cb00-e09d-418f-9ae9-b4fa378d93d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.479985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76cb00-e09d-418f-9ae9-b4fa378d93d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.480162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50790a1f-c4e4-43be-b851-2bfba6229b3e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.480273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76cb00-e09d-418f-9ae9-b4fa378d93d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.480523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76cb00-e09d-418f-9ae9-b4fa378d93d4 admin admin] PATH_INFO: `/v3/projects/0973c580aff64529a3d145ce7993efbe/users/69059809115e4619847bfbd07f643511/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.493597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1012/2031] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:18.497644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50790a1f-c4e4-43be-b851-2bfba6229b3e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.499390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76cb00-e09d-418f-9ae9-b4fa378d93d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0973c580aff64529a3d145ce7993efbe, user_id=69059809115e4619847bfbd07f643511, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.507905 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50790a1f-c4e4-43be-b851-2bfba6229b3e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:18.509395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76cb00-e09d-418f-9ae9-b4fa378d93d4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.513409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462eced-1895-4034-b0bc-228f3b7d2ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.514089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462eced-1895-4034-b0bc-228f3b7d2ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.514420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462eced-1895-4034-b0bc-228f3b7d2ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.514716 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462eced-1895-4034-b0bc-228f3b7d2ebe admin admin] PATH_INFO: `/v3/projects/18d3dc38a9fa4337b6c53661a21899a0/users/74ada4a3f2034c78b7412cafa511c340/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.531024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462eced-1895-4034-b0bc-228f3b7d2ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d3dc38a9fa4337b6c53661a21899a0, user_id=74ada4a3f2034c78b7412cafa511c340, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.534135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462eced-1895-4034-b0bc-228f3b7d2ebe admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.594822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1020/2032] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:18 2022] PUT /identity/v3/projects/0973c580aff64529a3d145ce7993efbe/users/69059809115e4619847bfbd07f643511/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:18.598538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76cb00-e09d-418f-9ae9-b4fa378d93d4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.651681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1013/2033] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:18 2022] PUT /identity/v3/projects/18d3dc38a9fa4337b6c53661a21899a0/users/74ada4a3f2034c78b7412cafa511c340/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:18.662829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca7aa0-434f-411c-b8f8-1edd8f57649b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.663113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca7aa0-434f-411c-b8f8-1edd8f57649b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.663386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca7aa0-434f-411c-b8f8-1edd8f57649b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.664981 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76cb00-e09d-418f-9ae9-b4fa378d93d4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:18.692809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-45ca7aa0-434f-411c-b8f8-1edd8f57649b None None] MFA Rules not processed for user `74ada4a3f2034c78b7412cafa511c340`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:18.694061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-45ca7aa0-434f-411c-b8f8-1edd8f57649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:18.721736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ba61b-1d9d-44ee-9073-1aac7c31b654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.722346 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ba61b-1d9d-44ee-9073-1aac7c31b654 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.722625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ba61b-1d9d-44ee-9073-1aac7c31b654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.722895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ba61b-1d9d-44ee-9073-1aac7c31b654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.723582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ba61b-1d9d-44ee-9073-1aac7c31b654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.725237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ba61b-1d9d-44ee-9073-1aac7c31b654 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.739275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1021/2034] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:18.740652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ba61b-1d9d-44ee-9073-1aac7c31b654 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.750724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ba61b-1d9d-44ee-9073-1aac7c31b654 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:18.755524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b2228-1c38-4d0a-ae04-4955824e78e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.756176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b2228-1c38-4d0a-ae04-4955824e78e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.756453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b2228-1c38-4d0a-ae04-4955824e78e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.756723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b2228-1c38-4d0a-ae04-4955824e78e0 admin admin] PATH_INFO: `/v3/projects/0973c580aff64529a3d145ce7993efbe/users/69059809115e4619847bfbd07f643511/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.758652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b2228-1c38-4d0a-ae04-4955824e78e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0973c580aff64529a3d145ce7993efbe, user_id=69059809115e4619847bfbd07f643511, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.761461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b2228-1c38-4d0a-ae04-4955824e78e0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.763740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1014/2035] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:18.768281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca7aa0-434f-411c-b8f8-1edd8f57649b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.772369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1022/2036] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:18 2022] PUT /identity/v3/projects/0973c580aff64529a3d145ce7993efbe/users/69059809115e4619847bfbd07f643511/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:18.779025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca7aa0-434f-411c-b8f8-1edd8f57649b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:18.783270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05350f53-27cb-4035-8d49-f936f93147c8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.783552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05350f53-27cb-4035-8d49-f936f93147c8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.783824 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05350f53-27cb-4035-8d49-f936f93147c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.814121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-05350f53-27cb-4035-8d49-f936f93147c8 None None] MFA Rules not processed for user `69059809115e4619847bfbd07f643511`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:18.815985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-05350f53-27cb-4035-8d49-f936f93147c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:18.854975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2fea6-a41d-48e6-9e5b-0a3fc3126e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:18.855813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2fea6-a41d-48e6-9e5b-0a3fc3126e32 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:18.856281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2fea6-a41d-48e6-9e5b-0a3fc3126e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:18.856568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2fea6-a41d-48e6-9e5b-0a3fc3126e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:18.857241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2fea6-a41d-48e6-9e5b-0a3fc3126e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:18.859305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2fea6-a41d-48e6-9e5b-0a3fc3126e32 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:18.895345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1023/2037] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:18.908607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05350f53-27cb-4035-8d49-f936f93147c8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.910241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-05350f53-27cb-4035-8d49-f936f93147c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:18.955740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1015/2038] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:18 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:18.959880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2fea6-a41d-48e6-9e5b-0a3fc3126e32 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:18.977327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:18.977784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05350f53-27cb-4035-8d49-f936f93147c8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:19.028782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2fea6-a41d-48e6-9e5b-0a3fc3126e32 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:19.036958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233bdec-9d29-4005-99b1-81974f4c49dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:19.038089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233bdec-9d29-4005-99b1-81974f4c49dc admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:19.038584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233bdec-9d29-4005-99b1-81974f4c49dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:19.039040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233bdec-9d29-4005-99b1-81974f4c49dc admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:19.039851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233bdec-9d29-4005-99b1-81974f4c49dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:19.039998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb0e34-8055-4b9c-87a9-064fa725f49d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:19.040643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb0e34-8055-4b9c-87a9-064fa725f49d service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:19.040923 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb0e34-8055-4b9c-87a9-064fa725f49d service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:19.041220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb0e34-8055-4b9c-87a9-064fa725f49d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:19.043067 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233bdec-9d29-4005-99b1-81974f4c49dc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:19.080237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:19.109052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb0e34-8055-4b9c-87a9-064fa725f49d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:19.111306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb0e34-8055-4b9c-87a9-064fa725f49d service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:19.120947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1024/2039] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:19.143057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1016/2040] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:18 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:19.147140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb0e34-8055-4b9c-87a9-064fa725f49d service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:19.167432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb0e34-8055-4b9c-87a9-064fa725f49d service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:19.174348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db6320-6197-47d6-848d-e41d32e2bd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:19.174985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db6320-6197-47d6-848d-e41d32e2bd68 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:19.175325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db6320-6197-47d6-848d-e41d32e2bd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:19.175610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db6320-6197-47d6-848d-e41d32e2bd68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:19.182076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db6320-6197-47d6-848d-e41d32e2bd68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:19.182076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db6320-6197-47d6-848d-e41d32e2bd68 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:19.198479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1025/2041] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:19.199708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233bdec-9d29-4005-99b1-81974f4c49dc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:19.205960 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db6320-6197-47d6-848d-e41d32e2bd68 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:19.213905 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233bdec-9d29-4005-99b1-81974f4c49dc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:19.221235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c32afa-4fa4-440e-9731-964e8907e7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:19.222121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c32afa-4fa4-440e-9731-964e8907e7c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:19.222235 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db6320-6197-47d6-848d-e41d32e2bd68 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:19.222458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c32afa-4fa4-440e-9731-964e8907e7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:19.222782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c32afa-4fa4-440e-9731-964e8907e7c3 admin admin] PATH_INFO: `/v3/projects/e1a1f0d36fe04bf5985dfa345d2d0de2/users/b4b3dec510da4b509a65dd8d1e6c3718/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:19.236427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af244588-750d-4116-b50a-1858e5b71d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:19.237175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af244588-750d-4116-b50a-1858e5b71d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:19.237451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af244588-750d-4116-b50a-1858e5b71d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:19.237722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af244588-750d-4116-b50a-1858e5b71d1b admin admin] PATH_INFO: `/v3/users/99bb5e17e42b4f8eb1790a603b64bd1d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:19.239443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af244588-750d-4116-b50a-1858e5b71d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=99bb5e17e42b4f8eb1790a603b64bd1d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:19.240114 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c32afa-4fa4-440e-9731-964e8907e7c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a1f0d36fe04bf5985dfa345d2d0de2, user_id=b4b3dec510da4b509a65dd8d1e6c3718, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:19.241434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af244588-750d-4116-b50a-1858e5b71d1b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:19.243074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c32afa-4fa4-440e-9731-964e8907e7c3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:19.322511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1017/2042] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:19 2022] PUT /identity/v3/projects/e1a1f0d36fe04bf5985dfa345d2d0de2/users/b4b3dec510da4b509a65dd8d1e6c3718/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:19.326313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c32afa-4fa4-440e-9731-964e8907e7c3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:19.399649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c32afa-4fa4-440e-9731-964e8907e7c3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:19.405779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a957c-c947-423e-aa23-7e71c6c1b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:19.406113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a957c-c947-423e-aa23-7e71c6c1b598 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:19.406402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a957c-c947-423e-aa23-7e71c6c1b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:19.406682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a957c-c947-423e-aa23-7e71c6c1b598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:19.407884 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a957c-c947-423e-aa23-7e71c6c1b598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:19.409429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a957c-c947-423e-aa23-7e71c6c1b598 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:19.415966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-af244588-750d-4116-b50a-1858e5b71d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99bb5e17e42b4f8eb1790a603b64bd1d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:19.421756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1018/2043] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:19.426844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a957c-c947-423e-aa23-7e71c6c1b598 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:19.428870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-af244588-750d-4116-b50a-1858e5b71d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99bb5e17e42b4f8eb1790a603b64bd1d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:19.441834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a957c-c947-423e-aa23-7e71c6c1b598 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:19.447175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4abd0-61eb-4d4c-a302-52d0613ddbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:19.448082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4abd0-61eb-4d4c-a302-52d0613ddbb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:19.448361 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4abd0-61eb-4d4c-a302-52d0613ddbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:19.448665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4abd0-61eb-4d4c-a302-52d0613ddbb4 admin admin] PATH_INFO: `/v3/projects/e1a1f0d36fe04bf5985dfa345d2d0de2/users/b4b3dec510da4b509a65dd8d1e6c3718/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:19.450836 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4abd0-61eb-4d4c-a302-52d0613ddbb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a1f0d36fe04bf5985dfa345d2d0de2, user_id=b4b3dec510da4b509a65dd8d1e6c3718, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:19.453519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4abd0-61eb-4d4c-a302-52d0613ddbb4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:19.463442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:19.472943 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1019/2044] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:19 2022] PUT /identity/v3/projects/e1a1f0d36fe04bf5985dfa345d2d0de2/users/b4b3dec510da4b509a65dd8d1e6c3718/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:19.483040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664f9ab-c338-42de-8507-40514915b7c0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:19.483699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664f9ab-c338-42de-8507-40514915b7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:19.483940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664f9ab-c338-42de-8507-40514915b7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:19.494298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-af244588-750d-4116-b50a-1858e5b71d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99bb5e17e42b4f8eb1790a603b64bd1d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:19.499479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-af244588-750d-4116-b50a-1858e5b71d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99bb5e17e42b4f8eb1790a603b64bd1d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:19.512980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-af244588-750d-4116-b50a-1858e5b71d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99bb5e17e42b4f8eb1790a603b64bd1d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:19.523927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b664f9ab-c338-42de-8507-40514915b7c0 None None] MFA Rules not processed for user `b4b3dec510da4b509a65dd8d1e6c3718`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:19.525288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b664f9ab-c338-42de-8507-40514915b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:19.525624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1026/2045] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:19 2022] DELETE /identity/v3/users/99bb5e17e42b4f8eb1790a603b64bd1d => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:19.530290 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930004f-4f09-4668-a5d4-82bc6a1fea48 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:19.530779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930004f-4f09-4668-a5d4-82bc6a1fea48 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:19.531430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930004f-4f09-4668-a5d4-82bc6a1fea48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:19.576775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6930004f-4f09-4668-a5d4-82bc6a1fea48 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:19.578208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6930004f-4f09-4668-a5d4-82bc6a1fea48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:19.623902 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:19.651844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1020/2046] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:19.660507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664f9ab-c338-42de-8507-40514915b7c0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:19.662275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b664f9ab-c338-42de-8507-40514915b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:19.687847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1027/2047] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:19.693927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:19.695682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930004f-4f09-4668-a5d4-82bc6a1fea48 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:19.698135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6930004f-4f09-4668-a5d4-82bc6a1fea48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:19.746363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664f9ab-c338-42de-8507-40514915b7c0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:19.752191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc613c-ff9b-4e43-9034-db97c954ab1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:19.753007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc613c-ff9b-4e43-9034-db97c954ab1d service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:19.753333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc613c-ff9b-4e43-9034-db97c954ab1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:19.753612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc613c-ff9b-4e43-9034-db97c954ab1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:19.755534 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f4bc613c-ff9b-4e43-9034-db97c954ab1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:19.770406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930004f-4f09-4668-a5d4-82bc6a1fea48 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:19.774198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f548a7-5b7b-4897-82a0-546d315d5d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:19.774810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f548a7-5b7b-4897-82a0-546d315d5d98 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:19.775092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f548a7-5b7b-4897-82a0-546d315d5d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:19.775416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f548a7-5b7b-4897-82a0-546d315d5d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:19.789842 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f548a7-5b7b-4897-82a0-546d315d5d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:19.791841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f548a7-5b7b-4897-82a0-546d315d5d98 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:19.803260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1028/2048] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:19.835489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc613c-ff9b-4e43-9034-db97c954ab1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:19.836841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc613c-ff9b-4e43-9034-db97c954ab1d service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:19.858897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1021/2049] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:01:20.389886 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:20.469006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f548a7-5b7b-4897-82a0-546d315d5d98 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:20.471560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b4f548a7-5b7b-4897-82a0-546d315d5d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:20.480542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:20.490636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f548a7-5b7b-4897-82a0-546d315d5d98 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:20.513796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:20.514495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:20.514776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:20.515055 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] PATH_INFO: `/v3/projects/a9869a793456413fad427ac04df6b8bf` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:20.516219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9869a793456413fad427ac04df6b8bf)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:20.519135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:20.646614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9869a793456413fad427ac04df6b8bf', 'request_id': 'req-b4335e4d-00e1-4523-b438-605c1170e764'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:20.666032 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9869a793456413fad427ac04df6b8bf', 'request_id': 'req-b4335e4d-00e1-4523-b438-605c1170e764'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:20.674940 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9869a793456413fad427ac04df6b8bf', 'request_id': 'req-b4335e4d-00e1-4523-b438-605c1170e764'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:20.738238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] The token cache is being invalidate because project a9869a793456413fad427ac04df6b8bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:20.738525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:20.739551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:20.741524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1029/2050] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:20 2022] DELETE /identity/v3/projects/a9869a793456413fad427ac04df6b8bf => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:20.745397 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc613c-ff9b-4e43-9034-db97c954ab1d service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:20.764001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc613c-ff9b-4e43-9034-db97c954ab1d service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:20.884757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:20.885558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:20.885934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:20.886303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] PATH_INFO: `/v3/users/69059809115e4619847bfbd07f643511` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:20.887725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69059809115e4619847bfbd07f643511)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:20.889973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:21.196975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69059809115e4619847bfbd07f643511'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:21.198032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69059809115e4619847bfbd07f643511'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:21.199039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69059809115e4619847bfbd07f643511'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:21.296417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69059809115e4619847bfbd07f643511'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:21.319163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69059809115e4619847bfbd07f643511'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:21.341744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1022/2051] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:20 2022] DELETE /identity/v3/users/69059809115e4619847bfbd07f643511 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:21.754793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:21.851818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:21.974243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:21.979262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:22.083370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4335e4d-00e1-4523-b438-605c1170e764 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:22.141799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:22.147342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:22.171086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e148f-20c4-4565-8a73-f3a2668aa706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:22.171957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e148f-20c4-4565-8a73-f3a2668aa706 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:22.172436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e148f-20c4-4565-8a73-f3a2668aa706 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:22.173604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e148f-20c4-4565-8a73-f3a2668aa706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:22.178512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-366e148f-20c4-4565-8a73-f3a2668aa706 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:22.324528 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e148f-20c4-4565-8a73-f3a2668aa706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:22.325397 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb14b28-ca3e-49e7-860d-530589e91501 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:22.328124 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e148f-20c4-4565-8a73-f3a2668aa706 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:22.333600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:22.334578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:22.335867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:22.336262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] PATH_INFO: `/v3/projects/0973c580aff64529a3d145ce7993efbe` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:22.339072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0973c580aff64529a3d145ce7993efbe)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:22.341529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:22.356503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1030/2052] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:22.493415 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0973c580aff64529a3d145ce7993efbe', 'request_id': 'req-546bcd83-ecdc-42d9-9d53-161a6f885a6d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:22.502325 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0973c580aff64529a3d145ce7993efbe', 'request_id': 'req-546bcd83-ecdc-42d9-9d53-161a6f885a6d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:22.512344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0973c580aff64529a3d145ce7993efbe', 'request_id': 'req-546bcd83-ecdc-42d9-9d53-161a6f885a6d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:22.621693 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] The token cache is being invalidate because project 0973c580aff64529a3d145ce7993efbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:22.622260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:22.623254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-546bcd83-ecdc-42d9-9d53-161a6f885a6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:22.625005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1023/2053] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:22 2022] DELETE /identity/v3/projects/0973c580aff64529a3d145ce7993efbe => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:22.630803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7712ce3-3c88-4b21-81e1-0ff96b30a098 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:22.631322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7712ce3-3c88-4b21-81e1-0ff96b30a098 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:22.631741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7712ce3-3c88-4b21-81e1-0ff96b30a098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:22.655420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c7712ce3-3c88-4b21-81e1-0ff96b30a098 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:22.658652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c7712ce3-3c88-4b21-81e1-0ff96b30a098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:22.719641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1031/2054] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:22.728112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a29ded-abb1-44ae-b2e6-c55129106731 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:22.728400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a29ded-abb1-44ae-b2e6-c55129106731 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:22.728673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a29ded-abb1-44ae-b2e6-c55129106731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:22.748555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-04a29ded-abb1-44ae-b2e6-c55129106731 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:22.749915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-04a29ded-abb1-44ae-b2e6-c55129106731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:22.762331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1024/2055] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:22.764629 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7712ce3-3c88-4b21-81e1-0ff96b30a098 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:22.776936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7712ce3-3c88-4b21-81e1-0ff96b30a098 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:22.782156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a2207-fc54-4363-91dc-786d1821a435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:22.783482 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a2207-fc54-4363-91dc-786d1821a435 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:22.787018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a2207-fc54-4363-91dc-786d1821a435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:22.787018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a2207-fc54-4363-91dc-786d1821a435 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:22.787018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a2207-fc54-4363-91dc-786d1821a435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:22.787018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a2207-fc54-4363-91dc-786d1821a435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:22.787018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a2207-fc54-4363-91dc-786d1821a435 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:22.803451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1032/2056] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:01:22.804804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a29ded-abb1-44ae-b2e6-c55129106731 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:22.814991 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a29ded-abb1-44ae-b2e6-c55129106731 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:22.820796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130387a-2394-498b-93a5-d160ef93e28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:22.821581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130387a-2394-498b-93a5-d160ef93e28c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:22.821963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130387a-2394-498b-93a5-d160ef93e28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:22.822305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130387a-2394-498b-93a5-d160ef93e28c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:22.823202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130387a-2394-498b-93a5-d160ef93e28c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:22.825288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130387a-2394-498b-93a5-d160ef93e28c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:22.914536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:22.938655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1025/2057] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:22 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:22.943700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a2207-fc54-4363-91dc-786d1821a435 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:22.954133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a2207-fc54-4363-91dc-786d1821a435 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:23.015892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3a187-36ca-4f3c-bb72-021948f5c2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:23.016821 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3a187-36ca-4f3c-bb72-021948f5c2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:23.017536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3a187-36ca-4f3c-bb72-021948f5c2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:23.017893 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3a187-36ca-4f3c-bb72-021948f5c2b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:23.018643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3a187-36ca-4f3c-bb72-021948f5c2b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:23.020752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3a187-36ca-4f3c-bb72-021948f5c2b1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:23.062784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:23.134637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1033/2058] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:22 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:23.138496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130387a-2394-498b-93a5-d160ef93e28c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:23.152853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130387a-2394-498b-93a5-d160ef93e28c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:23.165544 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7b9f2-d80c-4549-a510-d4e246ff82e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:23.166428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7b9f2-d80c-4549-a510-d4e246ff82e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:23.166857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7b9f2-d80c-4549-a510-d4e246ff82e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:23.167302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7b9f2-d80c-4549-a510-d4e246ff82e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:23.168100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7b9f2-d80c-4549-a510-d4e246ff82e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:23.169878 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7b9f2-d80c-4549-a510-d4e246ff82e4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:23.188878 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1026/2059] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:23.194586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3a187-36ca-4f3c-bb72-021948f5c2b1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:23.208035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3a187-36ca-4f3c-bb72-021948f5c2b1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:23.215657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784b189-9659-4e3c-9c1e-7feaaaa59392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:23.216276 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b189-9659-4e3c-9c1e-7feaaaa59392 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:23.216930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b189-9659-4e3c-9c1e-7feaaaa59392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:23.217056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784b189-9659-4e3c-9c1e-7feaaaa59392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:23.217820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784b189-9659-4e3c-9c1e-7feaaaa59392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:23.219427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784b189-9659-4e3c-9c1e-7feaaaa59392 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:23.236965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1034/2060] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:23.238323 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7b9f2-d80c-4549-a510-d4e246ff82e4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:23.250355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7b9f2-d80c-4549-a510-d4e246ff82e4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:23.271853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87dd958-b8fd-4e99-a80a-e0dfd0b58ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:23.272614 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dd958-b8fd-4e99-a80a-e0dfd0b58ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:23.272960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dd958-b8fd-4e99-a80a-e0dfd0b58ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:23.274153 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dd958-b8fd-4e99-a80a-e0dfd0b58ea3 admin admin] PATH_INFO: `/v3/projects/691438fdb04046939c21eb22c5fe0e42/users/ac1903ecf5974ecd87595e0419b494ac/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:23.304245 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87dd958-b8fd-4e99-a80a-e0dfd0b58ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=691438fdb04046939c21eb22c5fe0e42, user_id=ac1903ecf5974ecd87595e0419b494ac, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:23.307646 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87dd958-b8fd-4e99-a80a-e0dfd0b58ea3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:23.371949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1027/2061] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:23 2022] PUT /identity/v3/projects/691438fdb04046939c21eb22c5fe0e42/users/ac1903ecf5974ecd87595e0419b494ac/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:23.384290 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fdc1f-9d42-43cb-9bf4-43bbfd0cdef3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:23.385183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fdc1f-9d42-43cb-9bf4-43bbfd0cdef3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:23.385622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fdc1f-9d42-43cb-9bf4-43bbfd0cdef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:23.421414 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d15fdc1f-9d42-43cb-9bf4-43bbfd0cdef3 None None] MFA Rules not processed for user `ac1903ecf5974ecd87595e0419b494ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:23.422839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d15fdc1f-9d42-43cb-9bf4-43bbfd0cdef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:23.537771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1035/2062] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:23.542861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87dd958-b8fd-4e99-a80a-e0dfd0b58ea3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:23.697634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87dd958-b8fd-4e99-a80a-e0dfd0b58ea3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:23.697634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477eb4cb-2678-4dd8-a335-6f9d7e17a418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:23.697634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eb4cb-2678-4dd8-a335-6f9d7e17a418 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:23.697634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eb4cb-2678-4dd8-a335-6f9d7e17a418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:23.698691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eb4cb-2678-4dd8-a335-6f9d7e17a418 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:23.698691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477eb4cb-2678-4dd8-a335-6f9d7e17a418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:23.699441 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477eb4cb-2678-4dd8-a335-6f9d7e17a418 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:24.174530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1028/2063] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:23 2022] POST /identity/v3/projects => generated 387 bytes in 633 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:24.178198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fdc1f-9d42-43cb-9bf4-43bbfd0cdef3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:24.197294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fdc1f-9d42-43cb-9bf4-43bbfd0cdef3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:24.280125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57002-35be-4602-9660-076a6ae446ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:24.280777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57002-35be-4602-9660-076a6ae446ad admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:24.281085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57002-35be-4602-9660-076a6ae446ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:24.281361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57002-35be-4602-9660-076a6ae446ad admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:24.282074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e57002-35be-4602-9660-076a6ae446ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:24.285629 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e57002-35be-4602-9660-076a6ae446ad admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:24.400653 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1036/2064] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:24 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:24.404637 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477eb4cb-2678-4dd8-a335-6f9d7e17a418 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:24.417368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477eb4cb-2678-4dd8-a335-6f9d7e17a418 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:24.425196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae550966-2b45-4b86-a838-3780921735f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:24.425901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae550966-2b45-4b86-a838-3780921735f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:24.426125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae550966-2b45-4b86-a838-3780921735f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:24.426619 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae550966-2b45-4b86-a838-3780921735f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:24.427508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae550966-2b45-4b86-a838-3780921735f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:24.429459 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae550966-2b45-4b86-a838-3780921735f2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:24.443827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1029/2065] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:24.449765 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57002-35be-4602-9660-076a6ae446ad admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:24.461100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57002-35be-4602-9660-076a6ae446ad admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:24.461857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34073-c523-4347-8343-a4843bff5507 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:24.462432 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34073-c523-4347-8343-a4843bff5507 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:24.462721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e34073-c523-4347-8343-a4843bff5507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:24.466537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33223a26-918f-41bb-a707-82f7e45c1c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:24.467205 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33223a26-918f-41bb-a707-82f7e45c1c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:24.467491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33223a26-918f-41bb-a707-82f7e45c1c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:24.467767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33223a26-918f-41bb-a707-82f7e45c1c74 admin admin] PATH_INFO: `/v3/projects/f4571d911279440c85923fd928ca1433/users/d29a1b74a199488a85318e24c7a63be9/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:24.499766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33223a26-918f-41bb-a707-82f7e45c1c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4571d911279440c85923fd928ca1433, user_id=d29a1b74a199488a85318e24c7a63be9, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:24.502358 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33223a26-918f-41bb-a707-82f7e45c1c74 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:24.504636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f8e34073-c523-4347-8343-a4843bff5507 None None] MFA Rules not processed for user `6113630b377f40eda2f8994ae69633ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:24.506059 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f8e34073-c523-4347-8343-a4843bff5507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:24.575789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1030/2066] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:24.591046 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e34073-c523-4347-8343-a4843bff5507 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:24.592480 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f8e34073-c523-4347-8343-a4843bff5507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:24.613448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1037/2067] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:24 2022] PUT /identity/v3/projects/f4571d911279440c85923fd928ca1433/users/d29a1b74a199488a85318e24c7a63be9/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:24.620669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33223a26-918f-41bb-a707-82f7e45c1c74 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:24.674297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e34073-c523-4347-8343-a4843bff5507 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:24.683053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33223a26-918f-41bb-a707-82f7e45c1c74 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:24.687745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f33328-22d4-4f2c-a739-03ac81869fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:24.688345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f33328-22d4-4f2c-a739-03ac81869fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:24.689227 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f33328-22d4-4f2c-a739-03ac81869fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:24.689734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f33328-22d4-4f2c-a739-03ac81869fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:24.690187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f33328-22d4-4f2c-a739-03ac81869fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:24.692068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f33328-22d4-4f2c-a739-03ac81869fa6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:24.703951 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1038/2068] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:24.708083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f33328-22d4-4f2c-a739-03ac81869fa6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:24.720058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f33328-22d4-4f2c-a739-03ac81869fa6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:24.727198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e95d4-3e24-4c12-accb-d066721aee5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:24.728185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e95d4-3e24-4c12-accb-d066721aee5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:24.728488 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e95d4-3e24-4c12-accb-d066721aee5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:24.729545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e95d4-3e24-4c12-accb-d066721aee5d admin admin] PATH_INFO: `/v3/projects/f4571d911279440c85923fd928ca1433/users/d29a1b74a199488a85318e24c7a63be9/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:24.734579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e95d4-3e24-4c12-accb-d066721aee5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4571d911279440c85923fd928ca1433, user_id=d29a1b74a199488a85318e24c7a63be9, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:24.734579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e95d4-3e24-4c12-accb-d066721aee5d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:24.745213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1039/2069] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:24 2022] PUT /identity/v3/projects/f4571d911279440c85923fd928ca1433/users/d29a1b74a199488a85318e24c7a63be9/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:24.745486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a6b32-1d75-422d-bf19-ae268ed1d86d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:24.746103 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6b32-1d75-422d-bf19-ae268ed1d86d service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:24.746380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6b32-1d75-422d-bf19-ae268ed1d86d service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:24.746652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a6b32-1d75-422d-bf19-ae268ed1d86d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:24.755988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cdf63-8563-4529-bf32-c5d75c69e7d8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:24.756276 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cdf63-8563-4529-bf32-c5d75c69e7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:24.756555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cdf63-8563-4529-bf32-c5d75c69e7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:24.784630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f75cdf63-8563-4529-bf32-c5d75c69e7d8 None None] MFA Rules not processed for user `d29a1b74a199488a85318e24c7a63be9`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:24.785902 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f75cdf63-8563-4529-bf32-c5d75c69e7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:24.822936 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a6b32-1d75-422d-bf19-ae268ed1d86d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:24.824575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a6b32-1d75-422d-bf19-ae268ed1d86d service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:24.840523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1031/2070] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:01:24.869167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1040/2071] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:24.881385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a6b32-1d75-422d-bf19-ae268ed1d86d service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:24.883612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d89a6b32-1d75-422d-bf19-ae268ed1d86d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:24.904161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:24.948260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a6b32-1d75-422d-bf19-ae268ed1d86d service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:24.991050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:25.012156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9533d62-1c37-4b2b-9ad3-e940c1c478b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:25.012775 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9533d62-1c37-4b2b-9ad3-e940c1c478b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:25.013122 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9533d62-1c37-4b2b-9ad3-e940c1c478b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:25.013346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9533d62-1c37-4b2b-9ad3-e940c1c478b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:25.024018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9533d62-1c37-4b2b-9ad3-e940c1c478b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:25.025510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9533d62-1c37-4b2b-9ad3-e940c1c478b8 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:25.035819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1032/2072] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:26.225820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cdf63-8563-4529-bf32-c5d75c69e7d8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:26.237377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cdf63-8563-4529-bf32-c5d75c69e7d8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:26.239251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34920b-ebfa-4a08-b438-4e51a47dee4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:26.239855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34920b-ebfa-4a08-b438-4e51a47dee4a service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:26.240136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34920b-ebfa-4a08-b438-4e51a47dee4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:26.240405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34920b-ebfa-4a08-b438-4e51a47dee4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:26.242184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-fb34920b-ebfa-4a08-b438-4e51a47dee4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:26.303657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34920b-ebfa-4a08-b438-4e51a47dee4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:26.305284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34920b-ebfa-4a08-b438-4e51a47dee4a service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:26.315461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1041/2073] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:26.353495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:26.422787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:26.694494 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9533d62-1c37-4b2b-9ad3-e940c1c478b8 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:26.769648 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9533d62-1c37-4b2b-9ad3-e940c1c478b8 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:26.774641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:26.775290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:26.775574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:26.775850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] PATH_INFO: `/v3/users/ac1903ecf5974ecd87595e0419b494ac` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:26.776991 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1903ecf5974ecd87595e0419b494ac)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:26.780384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:26.800679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:27.109573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:27.119361 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1903ecf5974ecd87595e0419b494ac'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:27.121026 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1903ecf5974ecd87595e0419b494ac'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:27.121673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1903ecf5974ecd87595e0419b494ac'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:27.262938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1903ecf5974ecd87595e0419b494ac'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:27.292829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1903ecf5974ecd87595e0419b494ac'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:27.319586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1033/2074] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:26 2022] DELETE /identity/v3/users/ac1903ecf5974ecd87595e0419b494ac => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:27.319792 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff2842-9528-4e87-9c4d-8c0cc34ce5df None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:27.320233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff2842-9528-4e87-9c4d-8c0cc34ce5df None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:27.320340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff2842-9528-4e87-9c4d-8c0cc34ce5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:27.362180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-10ff2842-9528-4e87-9c4d-8c0cc34ce5df None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:27.363964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-10ff2842-9528-4e87-9c4d-8c0cc34ce5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:27.478024 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1042/2075] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:27.504763 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:27.506705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:27.597205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddb78d-7370-4a11-9d9f-12987b493aee admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:27.604604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f951b-1a17-4720-b3a2-5d442e57ce43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:27.605302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f951b-1a17-4720-b3a2-5d442e57ce43 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:27.605583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f951b-1a17-4720-b3a2-5d442e57ce43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:27.605852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f951b-1a17-4720-b3a2-5d442e57ce43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:27.617116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f951b-1a17-4720-b3a2-5d442e57ce43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:27.619279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f951b-1a17-4720-b3a2-5d442e57ce43 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:27.646232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1034/2076] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:27.908931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba305b-3ade-4cdf-ac56-b63a7d292d91 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:27.909468 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba305b-3ade-4cdf-ac56-b63a7d292d91 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:27.909747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba305b-3ade-4cdf-ac56-b63a7d292d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:27.937410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-22ba305b-3ade-4cdf-ac56-b63a7d292d91 None None] MFA Rules not processed for user `6113630b377f40eda2f8994ae69633ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:27.939381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-22ba305b-3ade-4cdf-ac56-b63a7d292d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:28.032590 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1043/2077] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:28.115591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:28.534520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f951b-1a17-4720-b3a2-5d442e57ce43 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:28.537188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c24f951b-1a17-4720-b3a2-5d442e57ce43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:28.561543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f951b-1a17-4720-b3a2-5d442e57ce43 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:28.575332 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba305b-3ade-4cdf-ac56-b63a7d292d91 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:28.577367 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:28.577781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-22ba305b-3ade-4cdf-ac56-b63a7d292d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:28.578097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:28.578385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:28.578704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] PATH_INFO: `/v3/projects/691438fdb04046939c21eb22c5fe0e42` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:28.580200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=691438fdb04046939c21eb22c5fe0e42)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:28.584489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:28.609308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba305b-3ade-4cdf-ac56-b63a7d292d91 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:28.618879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:28.619694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:28.619974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:28.620242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] PATH_INFO: `/v3/users/3065dc56483141108636d4f46c38bff1` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:28.621527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3065dc56483141108636d4f46c38bff1)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:28.629157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:28.747684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '691438fdb04046939c21eb22c5fe0e42', 'request_id': 'req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:28.754952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '691438fdb04046939c21eb22c5fe0e42', 'request_id': 'req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:28.762557 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '691438fdb04046939c21eb22c5fe0e42', 'request_id': 'req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:28.835003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] The token cache is being invalidate because project 691438fdb04046939c21eb22c5fe0e42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:28.841068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:28.841068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:28.841068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1035/2078] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:28 2022] DELETE /identity/v3/projects/691438fdb04046939c21eb22c5fe0e42 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:28.843796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:28.846389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:28.847607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3065dc56483141108636d4f46c38bff1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:28.855428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3065dc56483141108636d4f46c38bff1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:28.890809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:28.912164 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1dcf0-35cf-4ea1-bfb5-134ba858b6aa admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:28.916890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:28.924363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:28.924363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:28.924363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] PATH_INFO: `/v3/users/d29a1b74a199488a85318e24c7a63be9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:28.924363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d29a1b74a199488a85318e24c7a63be9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:28.924363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:28.947731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3065dc56483141108636d4f46c38bff1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:28.948382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3065dc56483141108636d4f46c38bff1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:28.961585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3065dc56483141108636d4f46c38bff1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:28.974060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1044/2079] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:28 2022] DELETE /identity/v3/users/3065dc56483141108636d4f46c38bff1 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:29.114077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd29a1b74a199488a85318e24c7a63be9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:29.114077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd29a1b74a199488a85318e24c7a63be9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:29.114077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd29a1b74a199488a85318e24c7a63be9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:29.190410 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd29a1b74a199488a85318e24c7a63be9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:29.203400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd29a1b74a199488a85318e24c7a63be9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:29.220338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1036/2080] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:28 2022] DELETE /identity/v3/users/d29a1b74a199488a85318e24c7a63be9 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:29.283268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:29.285661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:29.335106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:29.373593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba091e17-d1c3-42e3-b7e5-517b3f247938 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:29.438718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:29.440403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:29.440783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:29.442627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] PATH_INFO: `/v3/projects/e98a76881e9c41ec9429d3a71a8fd714` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:29.444743 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e98a76881e9c41ec9429d3a71a8fd714)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:29.447755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:29.568413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e98a76881e9c41ec9429d3a71a8fd714', 'request_id': 'req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:29.584933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e98a76881e9c41ec9429d3a71a8fd714', 'request_id': 'req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:29.602177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e98a76881e9c41ec9429d3a71a8fd714', 'request_id': 'req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:29.738499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:29.740011 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] The token cache is being invalidate because project e98a76881e9c41ec9429d3a71a8fd714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:29.740483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:29.746391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:29.746516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:29.746516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1045/2081] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:29 2022] DELETE /identity/v3/projects/e98a76881e9c41ec9429d3a71a8fd714 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:29.747572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:29.748975 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:29.756577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5251c-117e-4ffd-bdb3-d2c2e228a896 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:29.849150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0abc83-87ec-4c52-b02c-a4debd8c08f9 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:29.849150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ea201f-bce0-45a6-93ce-8f0fc4ac3d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:29.849870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea201f-bce0-45a6-93ce-8f0fc4ac3d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:29.850355 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea201f-bce0-45a6-93ce-8f0fc4ac3d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:29.850748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ea201f-bce0-45a6-93ce-8f0fc4ac3d43 admin admin] PATH_INFO: `/v3/users/a05a8b07966a4391982d877d93dcbefe` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:29.851995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea201f-bce0-45a6-93ce-8f0fc4ac3d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a05a8b07966a4391982d877d93dcbefe)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:29.854022 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ea201f-bce0-45a6-93ce-8f0fc4ac3d43 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:29.862735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:29.863372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:29.863655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:29.863923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] PATH_INFO: `/v3/projects/f4571d911279440c85923fd928ca1433` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:29.865382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4571d911279440c85923fd928ca1433)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:29.867275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:30.061866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-80ea201f-bce0-45a6-93ce-8f0fc4ac3d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a05a8b07966a4391982d877d93dcbefe'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.062368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4571d911279440c85923fd928ca1433', 'request_id': 'req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.070631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4571d911279440c85923fd928ca1433', 'request_id': 'req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.071419 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-80ea201f-bce0-45a6-93ce-8f0fc4ac3d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a05a8b07966a4391982d877d93dcbefe'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.077941 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4571d911279440c85923fd928ca1433', 'request_id': 'req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.138746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-80ea201f-bce0-45a6-93ce-8f0fc4ac3d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a05a8b07966a4391982d877d93dcbefe'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.140776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-80ea201f-bce0-45a6-93ce-8f0fc4ac3d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a05a8b07966a4391982d877d93dcbefe'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.160753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-80ea201f-bce0-45a6-93ce-8f0fc4ac3d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a05a8b07966a4391982d877d93dcbefe'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.167846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1046/2082] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:29 2022] DELETE /identity/v3/users/a05a8b07966a4391982d877d93dcbefe => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:30.218894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:30.225288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] The token cache is being invalidate because project f4571d911279440c85923fd928ca1433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:30.225614 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.226615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3517c4dd-4909-4dd1-9dca-48c3f2997ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.231196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1037/2083] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:29 2022] DELETE /identity/v3/projects/f4571d911279440c85923fd928ca1433 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:30.236940 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62347b-e942-45b6-a8ce-fe2561de2209 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.237252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62347b-e942-45b6-a8ce-fe2561de2209 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.237561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62347b-e942-45b6-a8ce-fe2561de2209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.260334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3e62347b-e942-45b6-a8ce-fe2561de2209 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:30.261752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3e62347b-e942-45b6-a8ce-fe2561de2209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:30.338513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1047/2084] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:30.347035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128f7f7-4029-4454-9b07-18d7a1403246 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.347327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128f7f7-4029-4454-9b07-18d7a1403246 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.347611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128f7f7-4029-4454-9b07-18d7a1403246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.372800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c128f7f7-4029-4454-9b07-18d7a1403246 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:30.372800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c128f7f7-4029-4454-9b07-18d7a1403246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:30.386015 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1038/2085] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:30.390428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62347b-e942-45b6-a8ce-fe2561de2209 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:30.400749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62347b-e942-45b6-a8ce-fe2561de2209 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:30.405498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91521dc-475b-4c48-be68-25014469c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:30.406110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91521dc-475b-4c48-be68-25014469c452 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.406395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91521dc-475b-4c48-be68-25014469c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.406671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91521dc-475b-4c48-be68-25014469c452 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.408112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91521dc-475b-4c48-be68-25014469c452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:30.408484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91521dc-475b-4c48-be68-25014469c452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:30.414656 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91521dc-475b-4c48-be68-25014469c452 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:30.437051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1048/2086] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:01:30.447737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128f7f7-4029-4454-9b07-18d7a1403246 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:30.448986 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128f7f7-4029-4454-9b07-18d7a1403246 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:30.451303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91521dc-475b-4c48-be68-25014469c452 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:30.453264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c91521dc-475b-4c48-be68-25014469c452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:30.460634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504215d-ed55-43b5-9050-f4e2e52ec340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:30.461322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504215d-ed55-43b5-9050-f4e2e52ec340 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.461607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504215d-ed55-43b5-9050-f4e2e52ec340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.461880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504215d-ed55-43b5-9050-f4e2e52ec340 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.463003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504215d-ed55-43b5-9050-f4e2e52ec340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:30.465665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504215d-ed55-43b5-9050-f4e2e52ec340 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:30.471557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91521dc-475b-4c48-be68-25014469c452 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:30.479345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:30.480094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.480443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.480820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] PATH_INFO: `/v3/projects/40e254cb662349b8bbf7ccf9ce709c33` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.482646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40e254cb662349b8bbf7ccf9ce709c33)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:30.484705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:30.613141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1039/2087] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:30 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:30.617187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504215d-ed55-43b5-9050-f4e2e52ec340 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:30.629606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40e254cb662349b8bbf7ccf9ce709c33', 'request_id': 'req-910ab42d-9f04-4301-989f-4571ab356838'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.635312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40e254cb662349b8bbf7ccf9ce709c33', 'request_id': 'req-910ab42d-9f04-4301-989f-4571ab356838'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.643824 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40e254cb662349b8bbf7ccf9ce709c33', 'request_id': 'req-910ab42d-9f04-4301-989f-4571ab356838'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.685686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504215d-ed55-43b5-9050-f4e2e52ec340 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:30.691655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93750a5c-bcab-4a50-8b83-cb9a23d51252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:30.692277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93750a5c-bcab-4a50-8b83-cb9a23d51252 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.692557 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93750a5c-bcab-4a50-8b83-cb9a23d51252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.692830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93750a5c-bcab-4a50-8b83-cb9a23d51252 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.693506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93750a5c-bcab-4a50-8b83-cb9a23d51252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:30.695452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93750a5c-bcab-4a50-8b83-cb9a23d51252 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:30.773233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:30.778745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] The token cache is being invalidate because project 40e254cb662349b8bbf7ccf9ce709c33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:30.779042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.779736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-910ab42d-9f04-4301-989f-4571ab356838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:30.781693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1049/2088] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:30 2022] DELETE /identity/v3/projects/40e254cb662349b8bbf7ccf9ce709c33 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:30.787235 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd2255-2471-47f8-82a4-e3174e7b0419 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.787884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd2255-2471-47f8-82a4-e3174e7b0419 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.788564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd2255-2471-47f8-82a4-e3174e7b0419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.807813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1040/2089] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:30 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:30.811880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93750a5c-bcab-4a50-8b83-cb9a23d51252 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:30.815076 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-93750a5c-bcab-4a50-8b83-cb9a23d51252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:30.832640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e1bd2255-2471-47f8-82a4-e3174e7b0419 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:30.834164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e1bd2255-2471-47f8-82a4-e3174e7b0419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:30.844107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93750a5c-bcab-4a50-8b83-cb9a23d51252 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:30.854268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1050/2090] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:30.854439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e48e51-1446-43e8-b606-e3ee2e82c878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:30.854439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e48e51-1446-43e8-b606-e3ee2e82c878 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.854439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e48e51-1446-43e8-b606-e3ee2e82c878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.854699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e48e51-1446-43e8-b606-e3ee2e82c878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.855221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e48e51-1446-43e8-b606-e3ee2e82c878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:30.856850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e48e51-1446-43e8-b606-e3ee2e82c878 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:30.858696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efb798-ac34-44cd-8c6f-2774ac9bcf1f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.858980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efb798-ac34-44cd-8c6f-2774ac9bcf1f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.859250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efb798-ac34-44cd-8c6f-2774ac9bcf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.868951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1041/2091] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:30.875609 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e48e51-1446-43e8-b606-e3ee2e82c878 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:30.889344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-01efb798-ac34-44cd-8c6f-2774ac9bcf1f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:30.890595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-01efb798-ac34-44cd-8c6f-2774ac9bcf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:30.893248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e48e51-1446-43e8-b606-e3ee2e82c878 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:30.898651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fdef4-51d5-45ea-8112-0882b0c4a4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:30.899852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fdef4-51d5-45ea-8112-0882b0c4a4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.900344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fdef4-51d5-45ea-8112-0882b0c4a4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.901203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fdef4-51d5-45ea-8112-0882b0c4a4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.901573 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1051/2092] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:30.902178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fdef4-51d5-45ea-8112-0882b0c4a4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:30.903822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fdef4-51d5-45ea-8112-0882b0c4a4d8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:30.903955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efb798-ac34-44cd-8c6f-2774ac9bcf1f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:30.916653 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efb798-ac34-44cd-8c6f-2774ac9bcf1f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:30.923019 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2d328-2a4a-49a2-9e9b-1ca385db93be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:30.923680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2d328-2a4a-49a2-9e9b-1ca385db93be admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.924992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2d328-2a4a-49a2-9e9b-1ca385db93be admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.924992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2d328-2a4a-49a2-9e9b-1ca385db93be admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.925259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2d328-2a4a-49a2-9e9b-1ca385db93be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:30.925499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2d328-2a4a-49a2-9e9b-1ca385db93be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:30.927110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2d328-2a4a-49a2-9e9b-1ca385db93be admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:30.932190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1042/2093] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:30.939635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fdef4-51d5-45ea-8112-0882b0c4a4d8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:30.961826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1052/2094] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:01:30.961964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fdef4-51d5-45ea-8112-0882b0c4a4d8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:30.963755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2d328-2a4a-49a2-9e9b-1ca385db93be admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:30.968495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81880fc8-af55-4ca4-87a2-9330196da6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:30.969494 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81880fc8-af55-4ca4-87a2-9330196da6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.969999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81880fc8-af55-4ca4-87a2-9330196da6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.970289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81880fc8-af55-4ca4-87a2-9330196da6c6 admin admin] PATH_INFO: `/v3/projects/5054d1e905c74581b171bd197fbd934e/users/5ea2b831b1204ddc9775353649872a0d/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.978098 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2d328-2a4a-49a2-9e9b-1ca385db93be admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:30.984390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9e053-03d1-4e05-a6bd-77e85ac6ffb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:30.985221 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9e053-03d1-4e05-a6bd-77e85ac6ffb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:30.985630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9e053-03d1-4e05-a6bd-77e85ac6ffb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:30.985989 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9e053-03d1-4e05-a6bd-77e85ac6ffb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:30.986831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9e053-03d1-4e05-a6bd-77e85ac6ffb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:30.988827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9e053-03d1-4e05-a6bd-77e85ac6ffb4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:30.992353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81880fc8-af55-4ca4-87a2-9330196da6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5054d1e905c74581b171bd197fbd934e, user_id=5ea2b831b1204ddc9775353649872a0d, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:30.997307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81880fc8-af55-4ca4-87a2-9330196da6c6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:31.117592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1043/2095] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:30 2022] PUT /identity/v3/projects/5054d1e905c74581b171bd197fbd934e/users/5ea2b831b1204ddc9775353649872a0d/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:31.119608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1bc72-a7e4-4009-a21d-bca2be904599 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:31.120069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1bc72-a7e4-4009-a21d-bca2be904599 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:31.120368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1bc72-a7e4-4009-a21d-bca2be904599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:31.127506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1053/2096] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:30 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:31.135294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9e053-03d1-4e05-a6bd-77e85ac6ffb4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:31.167981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-12e1bc72-a7e4-4009-a21d-bca2be904599 None None] MFA Rules not processed for user `5ea2b831b1204ddc9775353649872a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:31.169253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-12e1bc72-a7e4-4009-a21d-bca2be904599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:31.287595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9e053-03d1-4e05-a6bd-77e85ac6ffb4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:31.287691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1044/2097] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:31.287766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cffbbc-4b7f-4c83-89a3-25927508496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:31.287766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cffbbc-4b7f-4c83-89a3-25927508496b admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:31.287766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cffbbc-4b7f-4c83-89a3-25927508496b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:31.287766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cffbbc-4b7f-4c83-89a3-25927508496b admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:31.288036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cffbbc-4b7f-4c83-89a3-25927508496b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:31.288036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cffbbc-4b7f-4c83-89a3-25927508496b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:31.288164 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1bc72-a7e4-4009-a21d-bca2be904599 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:31.309331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1bc72-a7e4-4009-a21d-bca2be904599 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:31.315287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91156b5f-980f-4cdf-9157-d40b91620f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:31.322191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156b5f-980f-4cdf-9157-d40b91620f7f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:31.322191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156b5f-980f-4cdf-9157-d40b91620f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:31.322191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156b5f-980f-4cdf-9157-d40b91620f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:31.322191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91156b5f-980f-4cdf-9157-d40b91620f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:31.322191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91156b5f-980f-4cdf-9157-d40b91620f7f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:31.387033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1054/2098] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:31 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:31.387988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cffbbc-4b7f-4c83-89a3-25927508496b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:31.399669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cffbbc-4b7f-4c83-89a3-25927508496b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:31.405709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f55f5-d47b-49f3-b5d8-292d3efb3498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:31.406434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f55f5-d47b-49f3-b5d8-292d3efb3498 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:31.406720 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f55f5-d47b-49f3-b5d8-292d3efb3498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:31.407795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f55f5-d47b-49f3-b5d8-292d3efb3498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:31.408478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f55f5-d47b-49f3-b5d8-292d3efb3498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:31.410230 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f55f5-d47b-49f3-b5d8-292d3efb3498 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:31.422524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1055/2099] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:31.426743 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f55f5-d47b-49f3-b5d8-292d3efb3498 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:31.437621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f55f5-d47b-49f3-b5d8-292d3efb3498 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:31.443243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f79e2-ba16-4ca6-9568-00ab2f11525f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:31.443849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f79e2-ba16-4ca6-9568-00ab2f11525f admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:31.444131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f79e2-ba16-4ca6-9568-00ab2f11525f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:31.444408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f79e2-ba16-4ca6-9568-00ab2f11525f admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:31.446165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f79e2-ba16-4ca6-9568-00ab2f11525f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:31.446690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f79e2-ba16-4ca6-9568-00ab2f11525f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:31.462964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1056/2100] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:31.466216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f79e2-ba16-4ca6-9568-00ab2f11525f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:31.474402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1045/2101] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:31 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:31.479492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91156b5f-980f-4cdf-9157-d40b91620f7f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:31.479642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f79e2-ba16-4ca6-9568-00ab2f11525f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:31.546699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e6544-546b-4ee6-afab-9c4ab0bed7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:31.547738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e6544-546b-4ee6-afab-9c4ab0bed7e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:31.548722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e6544-546b-4ee6-afab-9c4ab0bed7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:31.549205 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e6544-546b-4ee6-afab-9c4ab0bed7e2 admin admin] PATH_INFO: `/v3/projects/a49771a58f2f4a8da8feaa68fe7be948/users/8a2eab4616b24d35b8fed9c54b276684/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:31.577246 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e6544-546b-4ee6-afab-9c4ab0bed7e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49771a58f2f4a8da8feaa68fe7be948, user_id=8a2eab4616b24d35b8fed9c54b276684, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:31.580487 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472e6544-546b-4ee6-afab-9c4ab0bed7e2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:31.602497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91156b5f-980f-4cdf-9157-d40b91620f7f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:31.608491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fc837-42f5-4ab8-b73a-bb584a86f09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:31.612102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fc837-42f5-4ab8-b73a-bb584a86f09d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:31.612227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fc837-42f5-4ab8-b73a-bb584a86f09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:31.612531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fc837-42f5-4ab8-b73a-bb584a86f09d admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:31.613292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fc837-42f5-4ab8-b73a-bb584a86f09d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:31.615534 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fc837-42f5-4ab8-b73a-bb584a86f09d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:31.750050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1057/2102] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:31 2022] PUT /identity/v3/projects/a49771a58f2f4a8da8feaa68fe7be948/users/8a2eab4616b24d35b8fed9c54b276684/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:31.754914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1bf2f-5f0a-4b96-9002-715f7e66a405 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:31.755413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1bf2f-5f0a-4b96-9002-715f7e66a405 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:31.757979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1bf2f-5f0a-4b96-9002-715f7e66a405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:31.769103 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1046/2103] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:31 2022] POST /identity/v3/users => generated 297 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:31.772571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fc837-42f5-4ab8-b73a-bb584a86f09d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:31.794842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fc837-42f5-4ab8-b73a-bb584a86f09d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:31.825444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-efe1bf2f-5f0a-4b96-9002-715f7e66a405 None None] MFA Rules not processed for user `8a2eab4616b24d35b8fed9c54b276684`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:31.826661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-efe1bf2f-5f0a-4b96-9002-715f7e66a405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:31.829739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:31.894320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb7266-98ef-463f-a483-a2a685e41322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:31.895306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb7266-98ef-463f-a483-a2a685e41322 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:31.895854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb7266-98ef-463f-a483-a2a685e41322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:31.896329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb7266-98ef-463f-a483-a2a685e41322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:31.896996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb7266-98ef-463f-a483-a2a685e41322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:31.898976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb7266-98ef-463f-a483-a2a685e41322 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:31.910497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1047/2104] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:31.918716 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb7266-98ef-463f-a483-a2a685e41322 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:31.929504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb7266-98ef-463f-a483-a2a685e41322 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:31.941354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59f054-3159-43d1-b4e2-299e516b8889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:31.946096 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59f054-3159-43d1-b4e2-299e516b8889 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:31.946861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59f054-3159-43d1-b4e2-299e516b8889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:31.947354 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59f054-3159-43d1-b4e2-299e516b8889 admin admin] PATH_INFO: `/v3/projects/cf98d418c6f5447a90f06fcf380ba062/users/eed54c20e9454b2f973ce5bc286ca48b/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:31.966084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59f054-3159-43d1-b4e2-299e516b8889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf98d418c6f5447a90f06fcf380ba062, user_id=eed54c20e9454b2f973ce5bc286ca48b, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:31.969271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59f054-3159-43d1-b4e2-299e516b8889 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:31.975128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1058/2105] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:31.979749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1bf2f-5f0a-4b96-9002-715f7e66a405 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:31.991344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1bf2f-5f0a-4b96-9002-715f7e66a405 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:32.004190 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e9764-8d9a-486d-8765-2af981814c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:32.004863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e9764-8d9a-486d-8765-2af981814c55 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:32.005447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e9764-8d9a-486d-8765-2af981814c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:32.005775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e9764-8d9a-486d-8765-2af981814c55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:32.006608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e9764-8d9a-486d-8765-2af981814c55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:32.008627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e9764-8d9a-486d-8765-2af981814c55 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:32.051347 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1048/2106] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:31 2022] PUT /identity/v3/projects/cf98d418c6f5447a90f06fcf380ba062/users/eed54c20e9454b2f973ce5bc286ca48b/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:32.055304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59f054-3159-43d1-b4e2-299e516b8889 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:32.111986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1059/2107] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:31 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:32.115978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e9764-8d9a-486d-8765-2af981814c55 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:32.131018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59f054-3159-43d1-b4e2-299e516b8889 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:32.205426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e9764-8d9a-486d-8765-2af981814c55 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:32.211333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bc2ab-0c95-4507-8436-5d8b7798a19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:32.219244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c6c03-c983-46eb-8280-13e1fe81a44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:32.219244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c6c03-c983-46eb-8280-13e1fe81a44c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:32.219444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c6c03-c983-46eb-8280-13e1fe81a44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:32.219818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c6c03-c983-46eb-8280-13e1fe81a44c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:32.220691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c6c03-c983-46eb-8280-13e1fe81a44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:32.222636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c6c03-c983-46eb-8280-13e1fe81a44c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:32.223768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bc2ab-0c95-4507-8436-5d8b7798a19c admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:32.224058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bc2ab-0c95-4507-8436-5d8b7798a19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:32.224340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bc2ab-0c95-4507-8436-5d8b7798a19c admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:32.224994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bc2ab-0c95-4507-8436-5d8b7798a19c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:32.237475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3bc2ab-0c95-4507-8436-5d8b7798a19c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:32.237715 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1049/2108] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:32.240202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c6c03-c983-46eb-8280-13e1fe81a44c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:32.268866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c6c03-c983-46eb-8280-13e1fe81a44c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:32.273372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2ac25-f49a-4123-9405-fcf87d830e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:32.274058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2ac25-f49a-4123-9405-fcf87d830e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:32.274341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2ac25-f49a-4123-9405-fcf87d830e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:32.274618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2ac25-f49a-4123-9405-fcf87d830e51 admin admin] PATH_INFO: `/v3/projects/cf98d418c6f5447a90f06fcf380ba062/users/eed54c20e9454b2f973ce5bc286ca48b/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:32.276585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2ac25-f49a-4123-9405-fcf87d830e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf98d418c6f5447a90f06fcf380ba062, user_id=eed54c20e9454b2f973ce5bc286ca48b, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:32.279221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2ac25-f49a-4123-9405-fcf87d830e51 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:32.295535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1050/2109] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:32 2022] PUT /identity/v3/projects/cf98d418c6f5447a90f06fcf380ba062/users/eed54c20e9454b2f973ce5bc286ca48b/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:32.304783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1060/2110] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:32 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:32.306270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69c134-d866-4f24-86e6-3c4356f3bedc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:32.306729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69c134-d866-4f24-86e6-3c4356f3bedc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:32.307012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69c134-d866-4f24-86e6-3c4356f3bedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:32.318553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bc2ab-0c95-4507-8436-5d8b7798a19c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:32.331908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bc2ab-0c95-4507-8436-5d8b7798a19c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:32.349293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ec69c134-d866-4f24-86e6-3c4356f3bedc None None] MFA Rules not processed for user `eed54c20e9454b2f973ce5bc286ca48b`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:32.352018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ec69c134-d866-4f24-86e6-3c4356f3bedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:32.386540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:32.409795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb9a72-84e3-4817-8642-161a9df46bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:32.410598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb9a72-84e3-4817-8642-161a9df46bde admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:32.410986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb9a72-84e3-4817-8642-161a9df46bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:32.411351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb9a72-84e3-4817-8642-161a9df46bde admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:32.412293 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb9a72-84e3-4817-8642-161a9df46bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:32.414131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb9a72-84e3-4817-8642-161a9df46bde admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:32.425754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1061/2111] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:32.430153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb9a72-84e3-4817-8642-161a9df46bde admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:32.442181 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb9a72-84e3-4817-8642-161a9df46bde admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:32.448521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2419e5-a408-4bd3-bed7-64134af1bcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:32.449220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2419e5-a408-4bd3-bed7-64134af1bcf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:32.449625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2419e5-a408-4bd3-bed7-64134af1bcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:32.449752 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2419e5-a408-4bd3-bed7-64134af1bcf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:32.450435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1051/2112] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:32.450566 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2419e5-a408-4bd3-bed7-64134af1bcf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:32.451993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2419e5-a408-4bd3-bed7-64134af1bcf3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:32.464913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1062/2113] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:32.468314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69c134-d866-4f24-86e6-3c4356f3bedc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:32.469152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2419e5-a408-4bd3-bed7-64134af1bcf3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:32.470507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ec69c134-d866-4f24-86e6-3c4356f3bedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:32.480611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2419e5-a408-4bd3-bed7-64134af1bcf3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:32.486676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579a869-d4a4-4f2e-9cc3-eea2ece5ed68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:32.487348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579a869-d4a4-4f2e-9cc3-eea2ece5ed68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:32.487843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579a869-d4a4-4f2e-9cc3-eea2ece5ed68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:32.487974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579a869-d4a4-4f2e-9cc3-eea2ece5ed68 admin admin] PATH_INFO: `/v3/projects/4d1b5fa350c4423eb9d5216a29020697/users/438d64a9a24e4c5ea1a0f8b1076a8a27/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:32.505628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579a869-d4a4-4f2e-9cc3-eea2ece5ed68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1b5fa350c4423eb9d5216a29020697, user_id=438d64a9a24e4c5ea1a0f8b1076a8a27, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:32.508182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579a869-d4a4-4f2e-9cc3-eea2ece5ed68 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:32.550526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69c134-d866-4f24-86e6-3c4356f3bedc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:32.558411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f896b-f92f-46c6-8b43-68f30c4923d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:32.558411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f896b-f92f-46c6-8b43-68f30c4923d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:32.558411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f896b-f92f-46c6-8b43-68f30c4923d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:32.558411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f896b-f92f-46c6-8b43-68f30c4923d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:32.567008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f896b-f92f-46c6-8b43-68f30c4923d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:32.568422 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f896b-f92f-46c6-8b43-68f30c4923d8 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:32.578264 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1052/2114] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:32.646188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1063/2115] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:32 2022] PUT /identity/v3/projects/4d1b5fa350c4423eb9d5216a29020697/users/438d64a9a24e4c5ea1a0f8b1076a8a27/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:32.647890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b17fed-13c3-46d6-b179-1830ca75c197 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:32.648342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b17fed-13c3-46d6-b179-1830ca75c197 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:32.648646 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b17fed-13c3-46d6-b179-1830ca75c197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:32.685809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-67b17fed-13c3-46d6-b179-1830ca75c197 None None] MFA Rules not processed for user `438d64a9a24e4c5ea1a0f8b1076a8a27`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:32.688492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-67b17fed-13c3-46d6-b179-1830ca75c197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:32.792349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1053/2116] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:32.804052 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579a869-d4a4-4f2e-9cc3-eea2ece5ed68 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:32.884525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579a869-d4a4-4f2e-9cc3-eea2ece5ed68 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:32.890424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1c015-a6d7-4e06-b5bf-ff43cf713f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:32.891292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1c015-a6d7-4e06-b5bf-ff43cf713f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:32.891781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1c015-a6d7-4e06-b5bf-ff43cf713f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:32.892071 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1c015-a6d7-4e06-b5bf-ff43cf713f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:32.892869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1c015-a6d7-4e06-b5bf-ff43cf713f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:32.894836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1c015-a6d7-4e06-b5bf-ff43cf713f06 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:33.005224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1064/2117] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:32 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:33.009547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b17fed-13c3-46d6-b179-1830ca75c197 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:33.023872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b17fed-13c3-46d6-b179-1830ca75c197 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:33.058769 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:33.106047 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b2673-aa3f-4389-914f-7c0d20ec886d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:33.106940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b2673-aa3f-4389-914f-7c0d20ec886d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:33.107608 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b2673-aa3f-4389-914f-7c0d20ec886d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:33.108043 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b2673-aa3f-4389-914f-7c0d20ec886d admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:33.112079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b2673-aa3f-4389-914f-7c0d20ec886d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:33.114209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b2673-aa3f-4389-914f-7c0d20ec886d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:33.211274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1054/2118] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:33 2022] POST /identity/v3/users => generated 299 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:33.214566 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1c015-a6d7-4e06-b5bf-ff43cf713f06 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:33.229259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1c015-a6d7-4e06-b5bf-ff43cf713f06 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:33.236220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a682bbe-d559-4a11-b662-dc69c5c835bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:33.236953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a682bbe-d559-4a11-b662-dc69c5c835bd admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:33.237283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a682bbe-d559-4a11-b662-dc69c5c835bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:33.237482 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a682bbe-d559-4a11-b662-dc69c5c835bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:33.238059 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a682bbe-d559-4a11-b662-dc69c5c835bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:33.239654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a682bbe-d559-4a11-b662-dc69c5c835bd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:33.254149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1065/2119] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:33.255555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b2673-aa3f-4389-914f-7c0d20ec886d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:33.266226 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b2673-aa3f-4389-914f-7c0d20ec886d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:33.272054 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb9ad6-4bd9-4a55-addc-f9f80108a47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:33.272703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb9ad6-4bd9-4a55-addc-f9f80108a47c admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:33.273071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb9ad6-4bd9-4a55-addc-f9f80108a47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:33.273360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb9ad6-4bd9-4a55-addc-f9f80108a47c admin admin] PATH_INFO: `/v3/projects/925fa167b91c4a428cf038ff5e9b891c/users/82678ec0ee9349aa8ef8690cc0cce71e/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:33.290580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb9ad6-4bd9-4a55-addc-f9f80108a47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=925fa167b91c4a428cf038ff5e9b891c, user_id=82678ec0ee9349aa8ef8690cc0cce71e, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:33.293376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb9ad6-4bd9-4a55-addc-f9f80108a47c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:33.400999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1055/2120] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:33 2022] PUT /identity/v3/projects/925fa167b91c4a428cf038ff5e9b891c/users/82678ec0ee9349aa8ef8690cc0cce71e/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:33.416009 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a682bbe-d559-4a11-b662-dc69c5c835bd admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:33.435716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a682bbe-d559-4a11-b662-dc69c5c835bd admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:33.505969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:33.515808 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a46140-82fb-49d0-8d22-f2496b90d96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:33.517536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a46140-82fb-49d0-8d22-f2496b90d96f admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:33.518075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a46140-82fb-49d0-8d22-f2496b90d96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:33.518442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a46140-82fb-49d0-8d22-f2496b90d96f admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:33.519199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a46140-82fb-49d0-8d22-f2496b90d96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:33.521098 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a46140-82fb-49d0-8d22-f2496b90d96f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:33.536164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1066/2121] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:33.540942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb9ad6-4bd9-4a55-addc-f9f80108a47c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:33.564971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb9ad6-4bd9-4a55-addc-f9f80108a47c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:33.568639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8b266-4e17-4ae7-9512-78f6aba03d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:33.569383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8b266-4e17-4ae7-9512-78f6aba03d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:33.569640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8b266-4e17-4ae7-9512-78f6aba03d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:33.570173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8b266-4e17-4ae7-9512-78f6aba03d45 admin admin] PATH_INFO: `/v3/projects/925fa167b91c4a428cf038ff5e9b891c/users/82678ec0ee9349aa8ef8690cc0cce71e/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:33.574225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8b266-4e17-4ae7-9512-78f6aba03d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=925fa167b91c4a428cf038ff5e9b891c, user_id=82678ec0ee9349aa8ef8690cc0cce71e, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:33.581047 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8b266-4e17-4ae7-9512-78f6aba03d45 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:33.603862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1056/2122] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:33 2022] PUT /identity/v3/projects/925fa167b91c4a428cf038ff5e9b891c/users/82678ec0ee9349aa8ef8690cc0cce71e/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:33.608364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25e1ba-2688-4832-8677-a50e7c7fbd7b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:33.608364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25e1ba-2688-4832-8677-a50e7c7fbd7b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:33.608364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25e1ba-2688-4832-8677-a50e7c7fbd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:33.639850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-6a25e1ba-2688-4832-8677-a50e7c7fbd7b None None] MFA Rules not processed for user `82678ec0ee9349aa8ef8690cc0cce71e`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:33.641136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6a25e1ba-2688-4832-8677-a50e7c7fbd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:33.733270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1067/2123] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:33.802202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8b266-4e17-4ae7-9512-78f6aba03d45 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:33.833182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25e1ba-2688-4832-8677-a50e7c7fbd7b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:33.855161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:33.912567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8b266-4e17-4ae7-9512-78f6aba03d45 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:33.916657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd8c63-ae21-4614-8757-48aeab38d068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:33.917409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8c63-ae21-4614-8757-48aeab38d068 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:33.917808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8c63-ae21-4614-8757-48aeab38d068 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:33.918160 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd8c63-ae21-4614-8757-48aeab38d068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:33.926161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25e1ba-2688-4832-8677-a50e7c7fbd7b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:33.932260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:33.932976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:33.933374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:33.934680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] PATH_INFO: `/v3/users/5ea2b831b1204ddc9775353649872a0d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:33.936362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea2b831b1204ddc9775353649872a0d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:33.938062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:33.997814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd8c63-ae21-4614-8757-48aeab38d068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:33.999401 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd8c63-ae21-4614-8757-48aeab38d068 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:34.009413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1057/2124] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:34.142327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea2b831b1204ddc9775353649872a0d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:34.150867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea2b831b1204ddc9775353649872a0d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:34.251361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea2b831b1204ddc9775353649872a0d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:34.252062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea2b831b1204ddc9775353649872a0d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:34.265182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea2b831b1204ddc9775353649872a0d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:34.271946 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1068/2125] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:33 2022] DELETE /identity/v3/users/5ea2b831b1204ddc9775353649872a0d => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:34.276332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44549d21-7f47-4c07-851f-9e67d1c8c828 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:34.276790 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44549d21-7f47-4c07-851f-9e67d1c8c828 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:34.276886 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44549d21-7f47-4c07-851f-9e67d1c8c828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:34.296196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-44549d21-7f47-4c07-851f-9e67d1c8c828 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:34.297462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-44549d21-7f47-4c07-851f-9e67d1c8c828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:34.366723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1058/2126] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:34.376847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:34.383964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:34.472089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcf9b8-a2fe-4c89-a79c-e9fde168034c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:34.478217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053abbce-b433-4111-9575-c443b48d2dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:34.478217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053abbce-b433-4111-9575-c443b48d2dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:34.478217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053abbce-b433-4111-9575-c443b48d2dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:34.478217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053abbce-b433-4111-9575-c443b48d2dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:34.488180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053abbce-b433-4111-9575-c443b48d2dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:34.489853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053abbce-b433-4111-9575-c443b48d2dd3 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:34.507441 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1069/2127] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:34.834136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:34.886237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:34.967582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44549d21-7f47-4c07-851f-9e67d1c8c828 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:34.969509 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-44549d21-7f47-4c07-851f-9e67d1c8c828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:34.972363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053abbce-b433-4111-9575-c443b48d2dd3 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:34.973997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-053abbce-b433-4111-9575-c443b48d2dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:34.981946 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44549d21-7f47-4c07-851f-9e67d1c8c828 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:34.987566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:34.988204 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:34.988479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:34.988911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] PATH_INFO: `/v3/projects/5054d1e905c74581b171bd197fbd934e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:34.990140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5054d1e905c74581b171bd197fbd934e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:34.992075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:34.992203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053abbce-b433-4111-9575-c443b48d2dd3 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:34.998861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca7356-efca-4f9b-8ec4-3e1b1ad1bf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:34.998861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca7356-efca-4f9b-8ec4-3e1b1ad1bf87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:34.998861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca7356-efca-4f9b-8ec4-3e1b1ad1bf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:34.998861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca7356-efca-4f9b-8ec4-3e1b1ad1bf87 admin admin] PATH_INFO: `/v3/users/3119f622d618465e9c8cb3a35aa2e065` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:34.999692 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca7356-efca-4f9b-8ec4-3e1b1ad1bf87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3119f622d618465e9c8cb3a35aa2e065)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:35.001898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca7356-efca-4f9b-8ec4-3e1b1ad1bf87 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:35.128083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5054d1e905c74581b171bd197fbd934e', 'request_id': 'req-b8752d07-125c-4a72-88fa-32304c81b2cb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.137679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5054d1e905c74581b171bd197fbd934e', 'request_id': 'req-b8752d07-125c-4a72-88fa-32304c81b2cb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.144620 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5054d1e905c74581b171bd197fbd934e', 'request_id': 'req-b8752d07-125c-4a72-88fa-32304c81b2cb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.214254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-68ca7356-efca-4f9b-8ec4-3e1b1ad1bf87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3119f622d618465e9c8cb3a35aa2e065'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.221263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-68ca7356-efca-4f9b-8ec4-3e1b1ad1bf87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3119f622d618465e9c8cb3a35aa2e065'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.239823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] The token cache is being invalidate because project 5054d1e905c74581b171bd197fbd934e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:35.240149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.242904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.242904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1059/2128] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:34 2022] DELETE /identity/v3/projects/5054d1e905c74581b171bd197fbd934e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:35.246560 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:35.247971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:35.317367 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8752d07-125c-4a72-88fa-32304c81b2cb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:35.322143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:35.322770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:35.323079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:35.323394 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] PATH_INFO: `/v3/users/eed54c20e9454b2f973ce5bc286ca48b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:35.324564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eed54c20e9454b2f973ce5bc286ca48b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:35.326490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:35.332134 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-68ca7356-efca-4f9b-8ec4-3e1b1ad1bf87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3119f622d618465e9c8cb3a35aa2e065'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.333224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-68ca7356-efca-4f9b-8ec4-3e1b1ad1bf87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3119f622d618465e9c8cb3a35aa2e065'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.350869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-68ca7356-efca-4f9b-8ec4-3e1b1ad1bf87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3119f622d618465e9c8cb3a35aa2e065'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.357307 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1070/2129] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:34 2022] DELETE /identity/v3/users/3119f622d618465e9c8cb3a35aa2e065 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:35.362349 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33055ee-a02f-451b-99a8-5d361a57ff70 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:35.362634 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33055ee-a02f-451b-99a8-5d361a57ff70 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:35.362929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33055ee-a02f-451b-99a8-5d361a57ff70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:35.381637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f33055ee-a02f-451b-99a8-5d361a57ff70 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:35.382888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f33055ee-a02f-451b-99a8-5d361a57ff70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:35.443744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1071/2130] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:35.453928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33055ee-a02f-451b-99a8-5d361a57ff70 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:35.455383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f33055ee-a02f-451b-99a8-5d361a57ff70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:35.480091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:35.514294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33055ee-a02f-451b-99a8-5d361a57ff70 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:35.517265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eed54c20e9454b2f973ce5bc286ca48b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.518338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eed54c20e9454b2f973ce5bc286ca48b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.518698 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eed54c20e9454b2f973ce5bc286ca48b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.518813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ae9a3-8b0f-45d2-aa11-9018a7041eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:35.519412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ae9a3-8b0f-45d2-aa11-9018a7041eaf service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:35.519934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ae9a3-8b0f-45d2-aa11-9018a7041eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:35.520377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ae9a3-8b0f-45d2-aa11-9018a7041eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:35.522128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:35.526010 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-925ae9a3-8b0f-45d2-aa11-9018a7041eaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:35.536115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ae9a3-8b0f-45d2-aa11-9018a7041eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:35.537575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ae9a3-8b0f-45d2-aa11-9018a7041eaf service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:35.546994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1072/2131] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:01:35.589140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eed54c20e9454b2f973ce5bc286ca48b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.602464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eed54c20e9454b2f973ce5bc286ca48b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:35.615303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1060/2132] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:35 2022] DELETE /identity/v3/users/eed54c20e9454b2f973ce5bc286ca48b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:35.713990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:36.129645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ae9a3-8b0f-45d2-aa11-9018a7041eaf service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:36.132009 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-925ae9a3-8b0f-45d2-aa11-9018a7041eaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:36.145563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:36.148208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:36.270753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ae9a3-8b0f-45d2-aa11-9018a7041eaf service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:36.275844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:36.301995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58432c94-3469-4004-8bca-c889f149144d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:36.310669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:36.311841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:36.312341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:36.312840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] PATH_INFO: `/v3/projects/399c4a938be94e549b963d3b4461a788` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:36.317386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=399c4a938be94e549b963d3b4461a788)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:36.319754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:36.373093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69807e42-dd7b-463b-ab40-c4650bec712b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:36.374335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807e42-dd7b-463b-ab40-c4650bec712b service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:36.374335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807e42-dd7b-463b-ab40-c4650bec712b service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:36.374781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807e42-dd7b-463b-ab40-c4650bec712b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:36.378059 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-69807e42-dd7b-463b-ab40-c4650bec712b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:36.493909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:36.522476 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69807e42-dd7b-463b-ab40-c4650bec712b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:36.524961 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69807e42-dd7b-463b-ab40-c4650bec712b service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:36.541314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399c4a938be94e549b963d3b4461a788', 'request_id': 'req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.541763 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1073/2133] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:36.543051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69807e42-dd7b-463b-ab40-c4650bec712b service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:36.544567 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-69807e42-dd7b-463b-ab40-c4650bec712b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:36.559316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69807e42-dd7b-463b-ab40-c4650bec712b service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:36.564062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399c4a938be94e549b963d3b4461a788', 'request_id': 'req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.573654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399c4a938be94e549b963d3b4461a788', 'request_id': 'req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.689860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:36.690506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:36.690779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:36.691050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] PATH_INFO: `/v3/projects/cf98d418c6f5447a90f06fcf380ba062` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:36.692189 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf98d418c6f5447a90f06fcf380ba062)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:36.694143 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:36.704838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] The token cache is being invalidate because project 399c4a938be94e549b963d3b4461a788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:36.705293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.706127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.708666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1061/2134] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:36 2022] DELETE /identity/v3/projects/399c4a938be94e549b963d3b4461a788 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:36.717313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:36.718876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:36.733651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc00c80-ef49-4e9a-882f-5d2c8a4bc902 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:36.740593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:36.740593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:36.740960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:36.741378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] PATH_INFO: `/v3/users/74ada4a3f2034c78b7412cafa511c340` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:36.742596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74ada4a3f2034c78b7412cafa511c340)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:36.744723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:36.807939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf98d418c6f5447a90f06fcf380ba062', 'request_id': 'req-b186dca7-82e4-4407-b654-61442770f658'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.816081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf98d418c6f5447a90f06fcf380ba062', 'request_id': 'req-b186dca7-82e4-4407-b654-61442770f658'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.822989 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf98d418c6f5447a90f06fcf380ba062', 'request_id': 'req-b186dca7-82e4-4407-b654-61442770f658'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.885541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] The token cache is being invalidate because project cf98d418c6f5447a90f06fcf380ba062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:36.885933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.886596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b186dca7-82e4-4407-b654-61442770f658 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.888333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1074/2135] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:36 2022] DELETE /identity/v3/projects/cf98d418c6f5447a90f06fcf380ba062 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:36.893753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afee46-9e49-47b9-b6f7-8288ddd80a8b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:36.894040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afee46-9e49-47b9-b6f7-8288ddd80a8b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:36.894313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afee46-9e49-47b9-b6f7-8288ddd80a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:36.913272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-11afee46-9e49-47b9-b6f7-8288ddd80a8b None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:36.914466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-11afee46-9e49-47b9-b6f7-8288ddd80a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:36.980038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1075/2136] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:36.985802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74ada4a3f2034c78b7412cafa511c340'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.986489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74ada4a3f2034c78b7412cafa511c340'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.987130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74ada4a3f2034c78b7412cafa511c340'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:36.988959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da7411-55b4-40ea-b15d-b7cc5866326e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:36.989282 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da7411-55b4-40ea-b15d-b7cc5866326e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:36.989560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da7411-55b4-40ea-b15d-b7cc5866326e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:37.011197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-99da7411-55b4-40ea-b15d-b7cc5866326e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:37.012873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-99da7411-55b4-40ea-b15d-b7cc5866326e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:37.022609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1076/2137] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:37.023616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da7411-55b4-40ea-b15d-b7cc5866326e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:37.024899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-99da7411-55b4-40ea-b15d-b7cc5866326e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:37.037133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da7411-55b4-40ea-b15d-b7cc5866326e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:37.041997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe41b8f-6be7-4f46-93f4-c74b51688e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:37.046168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41b8f-6be7-4f46-93f4-c74b51688e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:37.046168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41b8f-6be7-4f46-93f4-c74b51688e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:37.046168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe41b8f-6be7-4f46-93f4-c74b51688e40 admin admin] PATH_INFO: `/v3/users/617fcdadb5474920b1b0f1819000a5b7` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:37.046168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe41b8f-6be7-4f46-93f4-c74b51688e40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=617fcdadb5474920b1b0f1819000a5b7)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:37.046514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe41b8f-6be7-4f46-93f4-c74b51688e40 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:37.125046 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74ada4a3f2034c78b7412cafa511c340'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:37.140295 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74ada4a3f2034c78b7412cafa511c340'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:37.154228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1062/2138] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:36 2022] DELETE /identity/v3/users/74ada4a3f2034c78b7412cafa511c340 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:37.157360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:37.220617 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b31d8-9190-437f-a474-75a121d7efd3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:37.225275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd25ba2-4016-4288-9717-15253afd1b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:37.225889 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd25ba2-4016-4288-9717-15253afd1b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:37.226173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd25ba2-4016-4288-9717-15253afd1b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:37.226477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd25ba2-4016-4288-9717-15253afd1b63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:37.227134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd25ba2-4016-4288-9717-15253afd1b63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:37.227439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd25ba2-4016-4288-9717-15253afd1b63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:37.229085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd25ba2-4016-4288-9717-15253afd1b63 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:37.245659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1063/2139] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:01:37.246732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd25ba2-4016-4288-9717-15253afd1b63 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:37.257330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd25ba2-4016-4288-9717-15253afd1b63 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:37.262031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb27b85-7945-4dbd-b30a-085cf07d255a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:37.266409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb27b85-7945-4dbd-b30a-085cf07d255a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:37.266409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb27b85-7945-4dbd-b30a-085cf07d255a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:37.266409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb27b85-7945-4dbd-b30a-085cf07d255a admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:37.266409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb27b85-7945-4dbd-b30a-085cf07d255a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:37.266409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb27b85-7945-4dbd-b30a-085cf07d255a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:37.293183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fbe41b8f-6be7-4f46-93f4-c74b51688e40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '617fcdadb5474920b1b0f1819000a5b7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:37.300534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fbe41b8f-6be7-4f46-93f4-c74b51688e40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '617fcdadb5474920b1b0f1819000a5b7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:37.350617 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1064/2140] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:37 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:37.354959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb27b85-7945-4dbd-b30a-085cf07d255a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:37.388130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fbe41b8f-6be7-4f46-93f4-c74b51688e40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '617fcdadb5474920b1b0f1819000a5b7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:37.388769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fbe41b8f-6be7-4f46-93f4-c74b51688e40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '617fcdadb5474920b1b0f1819000a5b7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:37.424671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:37.424671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-fbe41b8f-6be7-4f46-93f4-c74b51688e40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '617fcdadb5474920b1b0f1819000a5b7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:37.430024 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1077/2141] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:37 2022] DELETE /identity/v3/users/617fcdadb5474920b1b0f1819000a5b7 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:37.432105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb27b85-7945-4dbd-b30a-085cf07d255a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:37.434936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ecbb0-5e81-4b93-9a2a-bea6d240bbd1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:37.434936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ecbb0-5e81-4b93-9a2a-bea6d240bbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:37.435146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ecbb0-5e81-4b93-9a2a-bea6d240bbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:37.474468 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ec4ecbb0-5e81-4b93-9a2a-bea6d240bbd1 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:37.475267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ec4ecbb0-5e81-4b93-9a2a-bea6d240bbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:37.503013 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555241ff-bebc-40fa-a56f-1a1ced5db1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:37.503759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555241ff-bebc-40fa-a56f-1a1ced5db1ba admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:37.504292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555241ff-bebc-40fa-a56f-1a1ced5db1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:37.504674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555241ff-bebc-40fa-a56f-1a1ced5db1ba admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:37.505452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555241ff-bebc-40fa-a56f-1a1ced5db1ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:37.507689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555241ff-bebc-40fa-a56f-1a1ced5db1ba admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:37.544554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1078/2142] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:37.554763 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ecbb0-5e81-4b93-9a2a-bea6d240bbd1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:37.558126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ec4ecbb0-5e81-4b93-9a2a-bea6d240bbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:37.594052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:37.615720 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ecbb0-5e81-4b93-9a2a-bea6d240bbd1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:37.619348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff364e1-fbc0-4614-b950-58db55c35209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:37.620797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff364e1-fbc0-4614-b950-58db55c35209 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:37.620797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff364e1-fbc0-4614-b950-58db55c35209 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:37.620797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff364e1-fbc0-4614-b950-58db55c35209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:37.633837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff364e1-fbc0-4614-b950-58db55c35209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:37.636491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff364e1-fbc0-4614-b950-58db55c35209 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:37.651772 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1079/2143] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:37.664004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:37.670737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1065/2144] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:37 2022] POST /identity/v3/users => generated 287 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:37.673341 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff364e1-fbc0-4614-b950-58db55c35209 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:37.675922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cff364e1-fbc0-4614-b950-58db55c35209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:37.679736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555241ff-bebc-40fa-a56f-1a1ced5db1ba admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:37.690820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-555241ff-bebc-40fa-a56f-1a1ced5db1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:37.696884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff364e1-fbc0-4614-b950-58db55c35209 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:37.706685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:37.709296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:37.709296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:37.709296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] PATH_INFO: `/v3/projects/18d3dc38a9fa4337b6c53661a21899a0` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:37.709296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18d3dc38a9fa4337b6c53661a21899a0)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:37.712628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:37.731005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555241ff-bebc-40fa-a56f-1a1ced5db1ba admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:37.764740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f12043-2e13-4ac8-b811-ee74c438a01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:37.766776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f12043-2e13-4ac8-b811-ee74c438a01c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:37.768739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f12043-2e13-4ac8-b811-ee74c438a01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:37.769916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f12043-2e13-4ac8-b811-ee74c438a01c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:37.771809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f12043-2e13-4ac8-b811-ee74c438a01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:37.776542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f12043-2e13-4ac8-b811-ee74c438a01c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:37.793256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1066/2145] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:37.794634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f12043-2e13-4ac8-b811-ee74c438a01c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:37.820489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f12043-2e13-4ac8-b811-ee74c438a01c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:37.828130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b27604-e2d2-43d8-a56c-82ed31ed4e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:37.829648 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b27604-e2d2-43d8-a56c-82ed31ed4e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:37.830105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b27604-e2d2-43d8-a56c-82ed31ed4e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:37.830676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b27604-e2d2-43d8-a56c-82ed31ed4e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:37.831591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b27604-e2d2-43d8-a56c-82ed31ed4e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:37.834373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b27604-e2d2-43d8-a56c-82ed31ed4e0a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:37.853089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1067/2146] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:37.854922 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b27604-e2d2-43d8-a56c-82ed31ed4e0a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:37.870574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b27604-e2d2-43d8-a56c-82ed31ed4e0a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:37.884594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18d3dc38a9fa4337b6c53661a21899a0', 'request_id': 'req-8dd2a920-5fca-41f9-948d-86b2b1588dd2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:37.899097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18d3dc38a9fa4337b6c53661a21899a0', 'request_id': 'req-8dd2a920-5fca-41f9-948d-86b2b1588dd2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:37.909239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18d3dc38a9fa4337b6c53661a21899a0', 'request_id': 'req-8dd2a920-5fca-41f9-948d-86b2b1588dd2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:37.962120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69357bae-fc21-439a-9246-5808775664dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:37.964373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69357bae-fc21-439a-9246-5808775664dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:37.964852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69357bae-fc21-439a-9246-5808775664dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:37.965329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69357bae-fc21-439a-9246-5808775664dc admin admin] PATH_INFO: `/v3/projects/36ef93a44ede4ca1875136e4973b1c64/users/58400d0dd0184722922beef7c637c6dc/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:37.990371 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69357bae-fc21-439a-9246-5808775664dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ef93a44ede4ca1875136e4973b1c64, user_id=58400d0dd0184722922beef7c637c6dc, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:37.993506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69357bae-fc21-439a-9246-5808775664dc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:38.030797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] The token cache is being invalidate because project 18d3dc38a9fa4337b6c53661a21899a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:38.031129 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.031807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.034143 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1080/2147] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:37 2022] DELETE /identity/v3/projects/18d3dc38a9fa4337b6c53661a21899a0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:38.037924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:38.039265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:38.053956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2a920-5fca-41f9-948d-86b2b1588dd2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:38.058909 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:38.059605 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:38.064591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:38.064591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] PATH_INFO: `/v3/users/b4b3dec510da4b509a65dd8d1e6c3718` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:38.064591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4b3dec510da4b509a65dd8d1e6c3718)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:38.064591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:38.091826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1068/2148] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:37 2022] PUT /identity/v3/projects/36ef93a44ede4ca1875136e4973b1c64/users/58400d0dd0184722922beef7c637c6dc/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:38.102585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac25f4b-655a-454b-8270-4f5439c5dd52 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:38.103018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac25f4b-655a-454b-8270-4f5439c5dd52 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:38.103373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac25f4b-655a-454b-8270-4f5439c5dd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:38.133959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8ac25f4b-655a-454b-8270-4f5439c5dd52 None None] MFA Rules not processed for user `58400d0dd0184722922beef7c637c6dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:38.135201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8ac25f4b-655a-454b-8270-4f5439c5dd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:38.243922 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1069/2149] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:38.245125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac25f4b-655a-454b-8270-4f5439c5dd52 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:38.246663 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8ac25f4b-655a-454b-8270-4f5439c5dd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:38.278955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4b3dec510da4b509a65dd8d1e6c3718'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.289628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4b3dec510da4b509a65dd8d1e6c3718'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.325719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac25f4b-655a-454b-8270-4f5439c5dd52 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:38.332102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:38.335840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:38.335840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:38.335840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] PATH_INFO: `/v3/projects/048d817dd3c54f1389c21eacb4a3a351` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:38.335840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=048d817dd3c54f1389c21eacb4a3a351)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:38.336360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:38.353751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4b3dec510da4b509a65dd8d1e6c3718'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.354388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4b3dec510da4b509a65dd8d1e6c3718'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.367530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4b3dec510da4b509a65dd8d1e6c3718'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.373369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1081/2150] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:38 2022] DELETE /identity/v3/users/b4b3dec510da4b509a65dd8d1e6c3718 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:38.377369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:38.378781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:38.439167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e128a5d-39b6-4747-a4aa-823c900764a8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:38.455375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048d817dd3c54f1389c21eacb4a3a351', 'request_id': 'req-b7693873-4da2-471d-8df7-897ba7ff7b3f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.461605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048d817dd3c54f1389c21eacb4a3a351', 'request_id': 'req-b7693873-4da2-471d-8df7-897ba7ff7b3f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.468731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048d817dd3c54f1389c21eacb4a3a351', 'request_id': 'req-b7693873-4da2-471d-8df7-897ba7ff7b3f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.491541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8da32-d12f-4a6f-9bfb-558de1e035bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:38.492140 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8da32-d12f-4a6f-9bfb-558de1e035bd service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:38.492428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8da32-d12f-4a6f-9bfb-558de1e035bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:38.492706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8da32-d12f-4a6f-9bfb-558de1e035bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:38.494504 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0bc8da32-d12f-4a6f-9bfb-558de1e035bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:38.557150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8da32-d12f-4a6f-9bfb-558de1e035bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:38.558641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8da32-d12f-4a6f-9bfb-558de1e035bd service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:38.569256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1082/2151] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:38.570145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8da32-d12f-4a6f-9bfb-558de1e035bd service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:38.571457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0bc8da32-d12f-4a6f-9bfb-558de1e035bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:38.588882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] The token cache is being invalidate because project 048d817dd3c54f1389c21eacb4a3a351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:38.588882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.589175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8da32-d12f-4a6f-9bfb-558de1e035bd service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:38.589799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:38.599055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1070/2152] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:38 2022] DELETE /identity/v3/projects/048d817dd3c54f1389c21eacb4a3a351 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:38.599201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351358a-7701-4a79-a20b-b5bfcdad5813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:38.599201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351358a-7701-4a79-a20b-b5bfcdad5813 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:38.599201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351358a-7701-4a79-a20b-b5bfcdad5813 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:38.599201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351358a-7701-4a79-a20b-b5bfcdad5813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:38.600827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:38.605416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3351358a-7701-4a79-a20b-b5bfcdad5813 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:38.606410 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:38.727416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351358a-7701-4a79-a20b-b5bfcdad5813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:38.729244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3351358a-7701-4a79-a20b-b5bfcdad5813 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:38.747318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7693873-4da2-471d-8df7-897ba7ff7b3f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:38.752812 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1083/2153] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:38.754876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df4ebe-c333-4cd1-9ba1-b4c200b0f973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:38.754876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df4ebe-c333-4cd1-9ba1-b4c200b0f973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:38.755135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df4ebe-c333-4cd1-9ba1-b4c200b0f973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:38.755326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df4ebe-c333-4cd1-9ba1-b4c200b0f973 admin admin] PATH_INFO: `/v3/users/58400d0dd0184722922beef7c637c6dc` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:38.757940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:38.759889 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df4ebe-c333-4cd1-9ba1-b4c200b0f973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58400d0dd0184722922beef7c637c6dc)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:38.763215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df4ebe-c333-4cd1-9ba1-b4c200b0f973 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:38.810995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:38.955710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351358a-7701-4a79-a20b-b5bfcdad5813 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:38.957592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3351358a-7701-4a79-a20b-b5bfcdad5813 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:38.995101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351358a-7701-4a79-a20b-b5bfcdad5813 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:38.998606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:38.999249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:38.999523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:38.999796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] PATH_INFO: `/v3/projects/e1a1f0d36fe04bf5985dfa345d2d0de2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.001630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a1f0d36fe04bf5985dfa345d2d0de2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:39.003603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:39.055705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-05df4ebe-c333-4cd1-9ba1-b4c200b0f973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58400d0dd0184722922beef7c637c6dc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:39.056439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-05df4ebe-c333-4cd1-9ba1-b4c200b0f973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58400d0dd0184722922beef7c637c6dc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:39.057410 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-05df4ebe-c333-4cd1-9ba1-b4c200b0f973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58400d0dd0184722922beef7c637c6dc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:39.159381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-05df4ebe-c333-4cd1-9ba1-b4c200b0f973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58400d0dd0184722922beef7c637c6dc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:39.174980 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-05df4ebe-c333-4cd1-9ba1-b4c200b0f973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58400d0dd0184722922beef7c637c6dc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:39.179807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a1f0d36fe04bf5985dfa345d2d0de2', 'request_id': 'req-b4a770a6-b887-4b16-81da-b68d3dba1a2c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:39.196829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1071/2154] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:38 2022] DELETE /identity/v3/users/58400d0dd0184722922beef7c637c6dc => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:39.200127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a1f0d36fe04bf5985dfa345d2d0de2', 'request_id': 'req-b4a770a6-b887-4b16-81da-b68d3dba1a2c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:39.201629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515eccd-2008-4fd1-b535-dedfe80b36a1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:39.201950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515eccd-2008-4fd1-b535-dedfe80b36a1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:39.202387 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515eccd-2008-4fd1-b535-dedfe80b36a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.209768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a1f0d36fe04bf5985dfa345d2d0de2', 'request_id': 'req-b4a770a6-b887-4b16-81da-b68d3dba1a2c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:39.223066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f515eccd-2008-4fd1-b535-dedfe80b36a1 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:39.224300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f515eccd-2008-4fd1-b535-dedfe80b36a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:39.296588 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1072/2155] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:39.298714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] The token cache is being invalidate because project e1a1f0d36fe04bf5985dfa345d2d0de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:39.299005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:39.299670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:39.301219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1084/2156] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:38 2022] DELETE /identity/v3/projects/e1a1f0d36fe04bf5985dfa345d2d0de2 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:39.306219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d23b9-6657-4365-8775-5880aea58d23 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:39.306388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:39.306510 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d23b9-6657-4365-8775-5880aea58d23 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:39.306788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d23b9-6657-4365-8775-5880aea58d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.307774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:39.326603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b40d23b9-6657-4365-8775-5880aea58d23 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:39.327718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b40d23b9-6657-4365-8775-5880aea58d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:39.336414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1073/2157] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:39.345581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320682f-ceed-4b96-93d0-613f26595ba9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:39.345868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320682f-ceed-4b96-93d0-613f26595ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:39.346169 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320682f-ceed-4b96-93d0-613f26595ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.365635 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a770a6-b887-4b16-81da-b68d3dba1a2c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:39.366517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9320682f-ceed-4b96-93d0-613f26595ba9 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:39.367722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9320682f-ceed-4b96-93d0-613f26595ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:39.376496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1074/2158] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:39.380669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320682f-ceed-4b96-93d0-613f26595ba9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:39.389652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320682f-ceed-4b96-93d0-613f26595ba9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:39.394974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f80a19-f010-451b-b88b-3c92ba82f491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:39.395578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f80a19-f010-451b-b88b-3c92ba82f491 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:39.395862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f80a19-f010-451b-b88b-3c92ba82f491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:39.396136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f80a19-f010-451b-b88b-3c92ba82f491 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.396781 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f80a19-f010-451b-b88b-3c92ba82f491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:39.397117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f80a19-f010-451b-b88b-3c92ba82f491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:39.398701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f80a19-f010-451b-b88b-3c92ba82f491 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:39.410878 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1075/2159] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:01:39.415122 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f80a19-f010-451b-b88b-3c92ba82f491 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:39.415858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c2322-f977-4b96-b7cc-6216281bfdf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:39.416592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c2322-f977-4b96-b7cc-6216281bfdf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:39.416879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c2322-f977-4b96-b7cc-6216281bfdf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:39.417200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c2322-f977-4b96-b7cc-6216281bfdf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.418988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-116c2322-f977-4b96-b7cc-6216281bfdf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:39.425257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f80a19-f010-451b-b88b-3c92ba82f491 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:39.429887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79db1bb-358c-45c3-8274-0467264d15c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:39.430469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79db1bb-358c-45c3-8274-0467264d15c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:39.430751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79db1bb-358c-45c3-8274-0467264d15c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:39.431024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79db1bb-358c-45c3-8274-0467264d15c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.431539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c2322-f977-4b96-b7cc-6216281bfdf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:39.431636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79db1bb-358c-45c3-8274-0467264d15c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:39.432962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116c2322-f977-4b96-b7cc-6216281bfdf8 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:39.433571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79db1bb-358c-45c3-8274-0467264d15c3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:39.443764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1085/2160] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:39.571186 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1076/2161] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:39 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:39.572557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c2322-f977-4b96-b7cc-6216281bfdf8 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:39.586969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c2322-f977-4b96-b7cc-6216281bfdf8 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:39.645836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c267a-27af-4b68-bd72-5cc6326fb0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:39.646444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c267a-27af-4b68-bd72-5cc6326fb0eb admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:39.647330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c267a-27af-4b68-bd72-5cc6326fb0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:39.647604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c267a-27af-4b68-bd72-5cc6326fb0eb admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.648267 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c267a-27af-4b68-bd72-5cc6326fb0eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:39.650360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c267a-27af-4b68-bd72-5cc6326fb0eb admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:39.803552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1086/2162] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:39 2022] POST /identity/v3/users => generated 286 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:39.806198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79db1bb-358c-45c3-8274-0467264d15c3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:39.824079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79db1bb-358c-45c3-8274-0467264d15c3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:39.829936 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a927da2-528d-4cb5-8f35-f686891fce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:39.830637 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a927da2-528d-4cb5-8f35-f686891fce7c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:39.831440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a927da2-528d-4cb5-8f35-f686891fce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:39.831440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a927da2-528d-4cb5-8f35-f686891fce7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.832016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a927da2-528d-4cb5-8f35-f686891fce7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:39.833756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a927da2-528d-4cb5-8f35-f686891fce7c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:39.859694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1077/2163] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:39.861343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c267a-27af-4b68-bd72-5cc6326fb0eb admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:39.886197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c267a-27af-4b68-bd72-5cc6326fb0eb admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:39.891123 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e02cc7-3945-4a0c-a404-5ca40607f1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:39.891893 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e02cc7-3945-4a0c-a404-5ca40607f1db admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:39.892598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e02cc7-3945-4a0c-a404-5ca40607f1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:39.892598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e02cc7-3945-4a0c-a404-5ca40607f1db admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.893324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e02cc7-3945-4a0c-a404-5ca40607f1db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:39.895118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e02cc7-3945-4a0c-a404-5ca40607f1db admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:39.910296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1087/2164] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:39.911186 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:39.912493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a927da2-528d-4cb5-8f35-f686891fce7c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:39.923754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a927da2-528d-4cb5-8f35-f686891fce7c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:39.928664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac4e32-17e6-4376-b793-70957be01227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:39.929774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac4e32-17e6-4376-b793-70957be01227 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:39.929774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac4e32-17e6-4376-b793-70957be01227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:39.930063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac4e32-17e6-4376-b793-70957be01227 admin admin] PATH_INFO: `/v3/projects/ec7b9cb55fc04b5c8a37774bbf8edc68/users/b44627b2e6ad473da289e0f954647155/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.941220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e02cc7-3945-4a0c-a404-5ca40607f1db admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:39.941877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e6e02cc7-3945-4a0c-a404-5ca40607f1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:39.949870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac4e32-17e6-4376-b793-70957be01227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7b9cb55fc04b5c8a37774bbf8edc68, user_id=b44627b2e6ad473da289e0f954647155, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:39.952953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac4e32-17e6-4376-b793-70957be01227 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:39.964542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e02cc7-3945-4a0c-a404-5ca40607f1db admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:39.972560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:39.973516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:39.973862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:39.974228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] PATH_INFO: `/v3/projects/36ef93a44ede4ca1875136e4973b1c64` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:39.976414 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ef93a44ede4ca1875136e4973b1c64)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:39.979560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:40.030231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1078/2165] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:39 2022] PUT /identity/v3/projects/ec7b9cb55fc04b5c8a37774bbf8edc68/users/b44627b2e6ad473da289e0f954647155/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:40.041464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93270c-b58a-4b96-8f78-3467d52c5c2a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:40.041928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93270c-b58a-4b96-8f78-3467d52c5c2a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:40.042272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93270c-b58a-4b96-8f78-3467d52c5c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:40.072765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4e93270c-b58a-4b96-8f78-3467d52c5c2a None None] MFA Rules not processed for user `b44627b2e6ad473da289e0f954647155`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:40.074081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4e93270c-b58a-4b96-8f78-3467d52c5c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:40.112984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ef93a44ede4ca1875136e4973b1c64', 'request_id': 'req-23a9bc31-b957-4a67-a185-29b72512884b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:40.126636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ef93a44ede4ca1875136e4973b1c64', 'request_id': 'req-23a9bc31-b957-4a67-a185-29b72512884b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:40.131747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ef93a44ede4ca1875136e4973b1c64', 'request_id': 'req-23a9bc31-b957-4a67-a185-29b72512884b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:40.161049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1079/2166] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:40.175825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93270c-b58a-4b96-8f78-3467d52c5c2a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:40.177464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4e93270c-b58a-4b96-8f78-3467d52c5c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:40.230892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] The token cache is being invalidate because project 36ef93a44ede4ca1875136e4973b1c64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:40.231426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:40.241272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-23a9bc31-b957-4a67-a185-29b72512884b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:40.245329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1088/2167] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:39 2022] DELETE /identity/v3/projects/36ef93a44ede4ca1875136e4973b1c64 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:40.248344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6ba8b-35d8-40fb-ac2b-a117786d7974 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:40.248696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6ba8b-35d8-40fb-ac2b-a117786d7974 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:40.249284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6ba8b-35d8-40fb-ac2b-a117786d7974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:40.268656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93270c-b58a-4b96-8f78-3467d52c5c2a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:40.274828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e9e6ba8b-35d8-40fb-ac2b-a117786d7974 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:40.276293 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e9e6ba8b-35d8-40fb-ac2b-a117786d7974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:40.332403 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f66d76-afff-4c37-8d8d-19587094c095 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83448e4c6704f7c8771132bd67eaa1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:40.333050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f66d76-afff-4c37-8d8d-19587094c095 service glance] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:40.333356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f66d76-afff-4c37-8d8d-19587094c095 service glance] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:40.333728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f66d76-afff-4c37-8d8d-19587094c095 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:40.335537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a8f66d76-afff-4c37-8d8d-19587094c095 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:40.362532 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1089/2168] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:40.369612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6790c-44ae-4e80-8b1f-4c7a6d631cac None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:40.370272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6790c-44ae-4e80-8b1f-4c7a6d631cac None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:40.370272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6790c-44ae-4e80-8b1f-4c7a6d631cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:40.391694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f4f6790c-44ae-4e80-8b1f-4c7a6d631cac None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:40.393062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f4f6790c-44ae-4e80-8b1f-4c7a6d631cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:40.401686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1090/2169] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:40.410642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6790c-44ae-4e80-8b1f-4c7a6d631cac None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:40.411991 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f66d76-afff-4c37-8d8d-19587094c095 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:40.413793 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f66d76-afff-4c37-8d8d-19587094c095 service glance] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:40.421761 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6790c-44ae-4e80-8b1f-4c7a6d631cac None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:40.424100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1080/2170] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:01:40.427034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb6f44-ecc2-47ab-bdb8-029dcd0627e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:40.427643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6f44-ecc2-47ab-bdb8-029dcd0627e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:40.427920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6f44-ecc2-47ab-bdb8-029dcd0627e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:40.428257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6f44-ecc2-47ab-bdb8-029dcd0627e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:40.428912 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb6f44-ecc2-47ab-bdb8-029dcd0627e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:40.429272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb6f44-ecc2-47ab-bdb8-029dcd0627e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:40.430849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb6f44-ecc2-47ab-bdb8-029dcd0627e2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:40.444201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1091/2171] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:01:40.450279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f66d76-afff-4c37-8d8d-19587094c095 service glance] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:40.474142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f66d76-afff-4c37-8d8d-19587094c095 service glance] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:40.479936 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b130b-3250-40f7-bd21-3e855c8294af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:40.481739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b130b-3250-40f7-bd21-3e855c8294af admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:40.482170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b130b-3250-40f7-bd21-3e855c8294af admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:40.482620 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b130b-3250-40f7-bd21-3e855c8294af admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:40.483542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b130b-3250-40f7-bd21-3e855c8294af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:40.485904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b130b-3250-40f7-bd21-3e855c8294af admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:40.492026 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:40.581942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1081/2172] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:40 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:01:40.586200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb6f44-ecc2-47ab-bdb8-029dcd0627e2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:40.596461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb6f44-ecc2-47ab-bdb8-029dcd0627e2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:40.655270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf8cfd-72bb-496c-a2d1-65c423ab44cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:40.655807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf8cfd-72bb-496c-a2d1-65c423ab44cd admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:40.656091 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf8cfd-72bb-496c-a2d1-65c423ab44cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:40.656374 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf8cfd-72bb-496c-a2d1-65c423ab44cd admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:40.657005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf8cfd-72bb-496c-a2d1-65c423ab44cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:40.658949 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf8cfd-72bb-496c-a2d1-65c423ab44cd admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:40.733114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1092/2173] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:40 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:40.737844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b130b-3250-40f7-bd21-3e855c8294af admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:40.751275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b130b-3250-40f7-bd21-3e855c8294af admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:40.756771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71596b8d-6c66-492b-bd10-954c41b30296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:40.757492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71596b8d-6c66-492b-bd10-954c41b30296 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:40.761525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71596b8d-6c66-492b-bd10-954c41b30296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:40.762736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71596b8d-6c66-492b-bd10-954c41b30296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:40.764171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71596b8d-6c66-492b-bd10-954c41b30296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:40.767324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71596b8d-6c66-492b-bd10-954c41b30296 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:40.779152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1082/2174] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:40.790016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf8cfd-72bb-496c-a2d1-65c423ab44cd admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:40.801630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf8cfd-72bb-496c-a2d1-65c423ab44cd admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:40.810594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c30a8-b2a2-433c-9513-d50bdb4c96c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:40.811185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c30a8-b2a2-433c-9513-d50bdb4c96c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:40.811466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c30a8-b2a2-433c-9513-d50bdb4c96c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:40.811742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c30a8-b2a2-433c-9513-d50bdb4c96c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:40.812657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c30a8-b2a2-433c-9513-d50bdb4c96c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:40.814900 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c30a8-b2a2-433c-9513-d50bdb4c96c7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:40.829565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1093/2175] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:40.832788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71596b8d-6c66-492b-bd10-954c41b30296 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:40.846106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71596b8d-6c66-492b-bd10-954c41b30296 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:40.852756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d4e6-22b9-48d5-8adf-d258ff54a34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:40.853480 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d4e6-22b9-48d5-8adf-d258ff54a34f admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:40.857067 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d4e6-22b9-48d5-8adf-d258ff54a34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:40.857067 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d4e6-22b9-48d5-8adf-d258ff54a34f admin admin] PATH_INFO: `/v3/projects/362da7f0f55a49289529a9fea00a42d3/users/129e4111f89c44fc8d46e70c16c8cc7e/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:40.871551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87d4e6-22b9-48d5-8adf-d258ff54a34f admin admin] RBAC: Authorizing `identity:create_grant(project_id=362da7f0f55a49289529a9fea00a42d3, user_id=129e4111f89c44fc8d46e70c16c8cc7e, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:40.874264 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87d4e6-22b9-48d5-8adf-d258ff54a34f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:41.024192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1083/2176] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:40 2022] PUT /identity/v3/projects/362da7f0f55a49289529a9fea00a42d3/users/129e4111f89c44fc8d46e70c16c8cc7e/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:41.036603 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96ce54-1368-411e-bde1-742974ce7d5c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:41.036888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96ce54-1368-411e-bde1-742974ce7d5c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:41.037188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96ce54-1368-411e-bde1-742974ce7d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:41.064325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7e96ce54-1368-411e-bde1-742974ce7d5c None None] MFA Rules not processed for user `129e4111f89c44fc8d46e70c16c8cc7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:41.065563 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7e96ce54-1368-411e-bde1-742974ce7d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:41.131069 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1094/2177] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:41.142285 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d4e6-22b9-48d5-8adf-d258ff54a34f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:41.143970 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2b87d4e6-22b9-48d5-8adf-d258ff54a34f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:41.203334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d4e6-22b9-48d5-8adf-d258ff54a34f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:41.207664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aee33e-d9c5-4da6-9260-16a1f5d5c6e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:41.208271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aee33e-d9c5-4da6-9260-16a1f5d5c6e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:41.208546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aee33e-d9c5-4da6-9260-16a1f5d5c6e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:41.208821 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aee33e-d9c5-4da6-9260-16a1f5d5c6e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:41.217251 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aee33e-d9c5-4da6-9260-16a1f5d5c6e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:41.218654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aee33e-d9c5-4da6-9260-16a1f5d5c6e3 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:41.227235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1084/2178] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:41.976747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:42.953871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:43.370565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:43.753747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:43.891691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96ce54-1368-411e-bde1-742974ce7d5c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:43.893260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7e96ce54-1368-411e-bde1-742974ce7d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:43.961333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96ce54-1368-411e-bde1-742974ce7d5c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:43.966349 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:43.966976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:43.967251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:43.967555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] PATH_INFO: `/v3/users/8a2eab4616b24d35b8fed9c54b276684` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:43.968694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a2eab4616b24d35b8fed9c54b276684)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:43.970729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:44.000334 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:44.230766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a2eab4616b24d35b8fed9c54b276684'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:44.237431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a2eab4616b24d35b8fed9c54b276684'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:44.360864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a2eab4616b24d35b8fed9c54b276684'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:44.361853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a2eab4616b24d35b8fed9c54b276684'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:44.382986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a2eab4616b24d35b8fed9c54b276684'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:44.389580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1095/2179] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:43 2022] DELETE /identity/v3/users/8a2eab4616b24d35b8fed9c54b276684 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:44.395596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae8fe-1850-4724-b3d1-01e5982be7e7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:44.396146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae8fe-1850-4724-b3d1-01e5982be7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:44.396240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ae8fe-1850-4724-b3d1-01e5982be7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:44.418966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c09ae8fe-1850-4724-b3d1-01e5982be7e7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:44.420357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c09ae8fe-1850-4724-b3d1-01e5982be7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:44.489480 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1085/2180] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:44.500938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:44.502622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:44.678275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e6dd1-2411-47f6-a709-215c121a45dc admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:44.683210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d39524-9461-4fc5-9179-6c329c2105c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:44.683833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d39524-9461-4fc5-9179-6c329c2105c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:44.684113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d39524-9461-4fc5-9179-6c329c2105c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:44.684383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d39524-9461-4fc5-9179-6c329c2105c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:44.709644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d39524-9461-4fc5-9179-6c329c2105c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:44.711708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d39524-9461-4fc5-9179-6c329c2105c2 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:44.726908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1096/2181] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:45.536104 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ae8fe-1850-4724-b3d1-01e5982be7e7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:45.538260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c09ae8fe-1850-4724-b3d1-01e5982be7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:45.549761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:45.554066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ae8fe-1850-4724-b3d1-01e5982be7e7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:45.559462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:45.560102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:45.560388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:45.560666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] PATH_INFO: `/v3/projects/a49771a58f2f4a8da8feaa68fe7be948` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:45.561951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49771a58f2f4a8da8feaa68fe7be948)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:45.564081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:45.705808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49771a58f2f4a8da8feaa68fe7be948', 'request_id': 'req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:45.711506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49771a58f2f4a8da8feaa68fe7be948', 'request_id': 'req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:45.718730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49771a58f2f4a8da8feaa68fe7be948', 'request_id': 'req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:45.851206 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] The token cache is being invalidate because project a49771a58f2f4a8da8feaa68fe7be948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:45.851206 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:45.851206 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:45.853046 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1086/2182] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:45 2022] DELETE /identity/v3/projects/a49771a58f2f4a8da8feaa68fe7be948 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:45.856587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d39524-9461-4fc5-9179-6c329c2105c2 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:45.869257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d39524-9461-4fc5-9179-6c329c2105c2 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:45.954337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:45.968841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:45.970200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:45.970484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:45.970760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] PATH_INFO: `/v3/users/438d64a9a24e4c5ea1a0f8b1076a8a27` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:45.972627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438d64a9a24e4c5ea1a0f8b1076a8a27)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:45.974676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:46.172726 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438d64a9a24e4c5ea1a0f8b1076a8a27'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:46.179868 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438d64a9a24e4c5ea1a0f8b1076a8a27'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:46.261878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438d64a9a24e4c5ea1a0f8b1076a8a27'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:46.262428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438d64a9a24e4c5ea1a0f8b1076a8a27'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:46.278562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438d64a9a24e4c5ea1a0f8b1076a8a27'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:46.287602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1097/2183] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:45 2022] DELETE /identity/v3/users/438d64a9a24e4c5ea1a0f8b1076a8a27 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:46.757376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:46.758974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:46.776146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfb9b3-c55e-40c9-a4df-77b305b37d2a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:46.980898 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:46.981545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:46.982066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:46.982503 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] PATH_INFO: `/v3/projects/4d1b5fa350c4423eb9d5216a29020697` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:46.984405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1b5fa350c4423eb9d5216a29020697)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:46.986445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:47.099019 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1b5fa350c4423eb9d5216a29020697', 'request_id': 'req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:47.104448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1b5fa350c4423eb9d5216a29020697', 'request_id': 'req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:47.114796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1b5fa350c4423eb9d5216a29020697', 'request_id': 'req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:47.215963 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] The token cache is being invalidate because project 4d1b5fa350c4423eb9d5216a29020697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:47.216288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:47.216915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:47.218434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1087/2184] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:46 2022] DELETE /identity/v3/projects/4d1b5fa350c4423eb9d5216a29020697 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:01:47.222963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:47.224402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:47.237854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed371c56-dc62-4f5a-8092-87334028a576 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:47.293266 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1da6b7-c193-4d1d-b314-336188a86dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:47.293891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1da6b7-c193-4d1d-b314-336188a86dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:47.294171 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1da6b7-c193-4d1d-b314-336188a86dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:47.294442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1da6b7-c193-4d1d-b314-336188a86dfc admin admin] PATH_INFO: `/v3/users/82678ec0ee9349aa8ef8690cc0cce71e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:47.295605 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1da6b7-c193-4d1d-b314-336188a86dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=82678ec0ee9349aa8ef8690cc0cce71e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:47.298186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1da6b7-c193-4d1d-b314-336188a86dfc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:47.441612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0d1da6b7-c193-4d1d-b314-336188a86dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82678ec0ee9349aa8ef8690cc0cce71e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:47.449701 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0d1da6b7-c193-4d1d-b314-336188a86dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82678ec0ee9349aa8ef8690cc0cce71e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:47.536617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0d1da6b7-c193-4d1d-b314-336188a86dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82678ec0ee9349aa8ef8690cc0cce71e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:47.537389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0d1da6b7-c193-4d1d-b314-336188a86dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82678ec0ee9349aa8ef8690cc0cce71e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:47.546967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:47.555541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0d1da6b7-c193-4d1d-b314-336188a86dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82678ec0ee9349aa8ef8690cc0cce71e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:47.562828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1098/2185] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:47 2022] DELETE /identity/v3/users/82678ec0ee9349aa8ef8690cc0cce71e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:48.058566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:48.060756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:48.079327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71d6e4-ffb7-4ac9-815b-ee522bb41197 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:48.097385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:48.160776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:48.161503 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:48.161782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:48.162057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] PATH_INFO: `/v3/projects/925fa167b91c4a428cf038ff5e9b891c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:48.163525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] RBAC: Authorizing `identity:delete_project(project_id=925fa167b91c4a428cf038ff5e9b891c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:48.166072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:48.317470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '925fa167b91c4a428cf038ff5e9b891c', 'request_id': 'req-6a2a6919-6492-4bf6-9958-c39f387b557f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:48.323247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '925fa167b91c4a428cf038ff5e9b891c', 'request_id': 'req-6a2a6919-6492-4bf6-9958-c39f387b557f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:48.331605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '925fa167b91c4a428cf038ff5e9b891c', 'request_id': 'req-6a2a6919-6492-4bf6-9958-c39f387b557f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:48.423738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] The token cache is being invalidate because project 925fa167b91c4a428cf038ff5e9b891c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:48.424288 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:48.424966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a2a6919-6492-4bf6-9958-c39f387b557f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:48.426749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1088/2186] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:48 2022] DELETE /identity/v3/projects/925fa167b91c4a428cf038ff5e9b891c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:48.432127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a901583-e82a-4e8c-bccb-f271d0706923 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:48.432639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a901583-e82a-4e8c-bccb-f271d0706923 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:48.436949 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a901583-e82a-4e8c-bccb-f271d0706923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:48.455477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2a901583-e82a-4e8c-bccb-f271d0706923 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:48.457782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2a901583-e82a-4e8c-bccb-f271d0706923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:48.542287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1099/2187] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:48.611112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a06c5a-6d39-4afa-89d5-458449c5af5f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:48.611733 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a06c5a-6d39-4afa-89d5-458449c5af5f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:48.612080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a06c5a-6d39-4afa-89d5-458449c5af5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:48.650865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-38a06c5a-6d39-4afa-89d5-458449c5af5f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:48.652668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-38a06c5a-6d39-4afa-89d5-458449c5af5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:48.662702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1089/2188] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:48.667694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a901583-e82a-4e8c-bccb-f271d0706923 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:48.688894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a901583-e82a-4e8c-bccb-f271d0706923 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:48.694493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed184e-7a63-4ccf-8d94-e88107dc2fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:48.695146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed184e-7a63-4ccf-8d94-e88107dc2fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:48.695537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed184e-7a63-4ccf-8d94-e88107dc2fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:48.695814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed184e-7a63-4ccf-8d94-e88107dc2fb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:48.696498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed184e-7a63-4ccf-8d94-e88107dc2fb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:48.696984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed184e-7a63-4ccf-8d94-e88107dc2fb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:48.698621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed184e-7a63-4ccf-8d94-e88107dc2fb7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:48.714746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1100/2189] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:01:48.719236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a06c5a-6d39-4afa-89d5-458449c5af5f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:48.728530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a06c5a-6d39-4afa-89d5-458449c5af5f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:48.733312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde2ca1-0ce0-4aa6-9e28-16903fb7bde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:48.733918 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde2ca1-0ce0-4aa6-9e28-16903fb7bde5 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:48.734195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde2ca1-0ce0-4aa6-9e28-16903fb7bde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:48.734470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde2ca1-0ce0-4aa6-9e28-16903fb7bde5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:48.735142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde2ca1-0ce0-4aa6-9e28-16903fb7bde5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:48.737071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde2ca1-0ce0-4aa6-9e28-16903fb7bde5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:48.819454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1090/2190] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:48 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:48.823783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed184e-7a63-4ccf-8d94-e88107dc2fb7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:48.835064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed184e-7a63-4ccf-8d94-e88107dc2fb7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:48.901112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6289d-e04b-40fc-89bb-5863f589b6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:48.901775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6289d-e04b-40fc-89bb-5863f589b6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:48.902178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6289d-e04b-40fc-89bb-5863f589b6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:48.902454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6289d-e04b-40fc-89bb-5863f589b6d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:48.903097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6289d-e04b-40fc-89bb-5863f589b6d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:48.905258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6289d-e04b-40fc-89bb-5863f589b6d8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:49.009147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1101/2191] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:48 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:49.010121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde2ca1-0ce0-4aa6-9e28-16903fb7bde5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:49.021137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde2ca1-0ce0-4aa6-9e28-16903fb7bde5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:49.032143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacc024-c3cd-4343-aad9-ffe7994452fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:49.032143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacc024-c3cd-4343-aad9-ffe7994452fd admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:49.032143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacc024-c3cd-4343-aad9-ffe7994452fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:49.032143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacc024-c3cd-4343-aad9-ffe7994452fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:49.032461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacc024-c3cd-4343-aad9-ffe7994452fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:49.032461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacc024-c3cd-4343-aad9-ffe7994452fd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:49.045067 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1091/2192] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:49.048234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6289d-e04b-40fc-89bb-5863f589b6d8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:49.058486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6289d-e04b-40fc-89bb-5863f589b6d8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:49.067233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f56cd7-9989-421c-93c2-8546f9cee44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:49.067233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f56cd7-9989-421c-93c2-8546f9cee44c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:49.067525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f56cd7-9989-421c-93c2-8546f9cee44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:49.067874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f56cd7-9989-421c-93c2-8546f9cee44c admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:49.068386 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f56cd7-9989-421c-93c2-8546f9cee44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:49.070022 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f56cd7-9989-421c-93c2-8546f9cee44c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:49.082666 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1102/2193] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:01:49.087776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacc024-c3cd-4343-aad9-ffe7994452fd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:49.099135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacc024-c3cd-4343-aad9-ffe7994452fd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:49.104226 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce39c7-3807-4904-a81d-8ac3b855bec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:49.105043 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce39c7-3807-4904-a81d-8ac3b855bec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:49.105398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce39c7-3807-4904-a81d-8ac3b855bec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:49.105981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce39c7-3807-4904-a81d-8ac3b855bec9 admin admin] PATH_INFO: `/v3/projects/7c94aa1e39a540fca4ff442e1b7f599a/users/61196d978fd3429caeceeda0bd9b9a90/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:49.130244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce39c7-3807-4904-a81d-8ac3b855bec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c94aa1e39a540fca4ff442e1b7f599a, user_id=61196d978fd3429caeceeda0bd9b9a90, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:49.133053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce39c7-3807-4904-a81d-8ac3b855bec9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:49.220886 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1092/2194] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:49 2022] PUT /identity/v3/projects/7c94aa1e39a540fca4ff442e1b7f599a/users/61196d978fd3429caeceeda0bd9b9a90/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:49.231990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295908a-bc44-44ef-a1a4-5861172bd8db None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:49.232484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295908a-bc44-44ef-a1a4-5861172bd8db None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:49.232575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295908a-bc44-44ef-a1a4-5861172bd8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:49.261933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7295908a-bc44-44ef-a1a4-5861172bd8db None None] MFA Rules not processed for user `61196d978fd3429caeceeda0bd9b9a90`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:49.263179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7295908a-bc44-44ef-a1a4-5861172bd8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:49.315884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:49.349962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1103/2195] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:01:49.366046 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce39c7-3807-4904-a81d-8ac3b855bec9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:49.367810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-69ce39c7-3807-4904-a81d-8ac3b855bec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:49.433920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:49.464228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce39c7-3807-4904-a81d-8ac3b855bec9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:49.473299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626475a5-1085-4002-959c-45115983edc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:49.474455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626475a5-1085-4002-959c-45115983edc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:49.474455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626475a5-1085-4002-959c-45115983edc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:49.474655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626475a5-1085-4002-959c-45115983edc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:49.485820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626475a5-1085-4002-959c-45115983edc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:49.487322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626475a5-1085-4002-959c-45115983edc6 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:49.501582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1093/2196] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:51.308358 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad725a6b-d6dd-49c6-a4ea-130149d9262f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:51.308918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad725a6b-d6dd-49c6-a4ea-130149d9262f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:51.308918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad725a6b-d6dd-49c6-a4ea-130149d9262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:51.330489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ad725a6b-d6dd-49c6-a4ea-130149d9262f None None] MFA Rules not processed for user `28b31650305943b498f02c882baa0144`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:51.332114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ad725a6b-d6dd-49c6-a4ea-130149d9262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:51.402315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1104/2197] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:52.119317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fbacc-805e-41a8-be41-daa60c2072ca None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:52.119805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fbacc-805e-41a8-be41-daa60c2072ca None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:52.119902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fbacc-805e-41a8-be41-daa60c2072ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:52.140093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d19fbacc-805e-41a8-be41-daa60c2072ca None None] MFA Rules not processed for user `c62f05ea732142c1af1c09dd23e7b9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:52.141947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d19fbacc-805e-41a8-be41-daa60c2072ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:52.231639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1094/2198] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:52.758740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad725a6b-d6dd-49c6-a4ea-130149d9262f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:52.827886 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad725a6b-d6dd-49c6-a4ea-130149d9262f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:52.833435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:52.834090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:52.834373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:52.834643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] PATH_INFO: `/v3/users/61196d978fd3429caeceeda0bd9b9a90` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:52.835768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61196d978fd3429caeceeda0bd9b9a90)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:52.837763 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:53.065231 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61196d978fd3429caeceeda0bd9b9a90'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:53.082776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61196d978fd3429caeceeda0bd9b9a90'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:53.221744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61196d978fd3429caeceeda0bd9b9a90'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:53.222669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61196d978fd3429caeceeda0bd9b9a90'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:53.236489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61196d978fd3429caeceeda0bd9b9a90'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:53.242696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1105/2199] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:52 2022] DELETE /identity/v3/users/61196d978fd3429caeceeda0bd9b9a90 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:53.247304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440e372-cf3b-4708-9fa0-9f5ad20a57ea None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:53.248127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440e372-cf3b-4708-9fa0-9f5ad20a57ea None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:53.248127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440e372-cf3b-4708-9fa0-9f5ad20a57ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:53.275525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2440e372-cf3b-4708-9fa0-9f5ad20a57ea None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:53.277400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2440e372-cf3b-4708-9fa0-9f5ad20a57ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:53.358185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1095/2200] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:53.368029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:53.369863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:53.413803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:53.465891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16417d7b-4c40-4bf9-9ac4-4a980d9d2580 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:53.470432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5bef0-ec8c-42d3-83b0-6655715815ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:53.471729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5bef0-ec8c-42d3-83b0-6655715815ca service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:53.472024 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5bef0-ec8c-42d3-83b0-6655715815ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:53.472296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5bef0-ec8c-42d3-83b0-6655715815ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:53.482262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5bef0-ec8c-42d3-83b0-6655715815ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:53.483785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5bef0-ec8c-42d3-83b0-6655715815ca service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:53.494007 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1106/2201] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:53.938799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440e372-cf3b-4708-9fa0-9f5ad20a57ea None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:53.940429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2440e372-cf3b-4708-9fa0-9f5ad20a57ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:53.952975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440e372-cf3b-4708-9fa0-9f5ad20a57ea None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:53.972753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:53.975539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:53.975539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:53.975539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] PATH_INFO: `/v3/projects/7c94aa1e39a540fca4ff442e1b7f599a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:53.975539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c94aa1e39a540fca4ff442e1b7f599a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:53.977315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:54.103368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c94aa1e39a540fca4ff442e1b7f599a', 'request_id': 'req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:54.110265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c94aa1e39a540fca4ff442e1b7f599a', 'request_id': 'req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:54.116753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c94aa1e39a540fca4ff442e1b7f599a', 'request_id': 'req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:54.211138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] The token cache is being invalidate because project 7c94aa1e39a540fca4ff442e1b7f599a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:54.211444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:54.212108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2a41125b-bc0d-44d4-80b0-a6fb6bb8367c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:54.213873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1096/2202] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:53 2022] DELETE /identity/v3/projects/7c94aa1e39a540fca4ff442e1b7f599a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:54.492632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5bef0-ec8c-42d3-83b0-6655715815ca service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:54.494866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-05d5bef0-ec8c-42d3-83b0-6655715815ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:54.510005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5bef0-ec8c-42d3-83b0-6655715815ca service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:54.574510 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:54.575169 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:54.575447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:54.575719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] PATH_INFO: `/v3/users/129e4111f89c44fc8d46e70c16c8cc7e` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:54.576923 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=129e4111f89c44fc8d46e70c16c8cc7e)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:54.579802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:55.206070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '129e4111f89c44fc8d46e70c16c8cc7e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:55.227690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '129e4111f89c44fc8d46e70c16c8cc7e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:55.319021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '129e4111f89c44fc8d46e70c16c8cc7e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:55.319669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '129e4111f89c44fc8d46e70c16c8cc7e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:55.334152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '129e4111f89c44fc8d46e70c16c8cc7e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:55.341651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1107/2203] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:01:54 2022] DELETE /identity/v3/users/129e4111f89c44fc8d46e70c16c8cc7e => generated 0 bytes in 850 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:55.346372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83df65-dc6c-446e-b255-ad9ffd7f2570 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:55.346920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83df65-dc6c-446e-b255-ad9ffd7f2570 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:55.346920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83df65-dc6c-446e-b255-ad9ffd7f2570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:55.367605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0b83df65-dc6c-446e-b255-ad9ffd7f2570 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:55.368922 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0b83df65-dc6c-446e-b255-ad9ffd7f2570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:55.379456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:55.433082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1097/2204] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:55.443240 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:55.444689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:55.496095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:55.543919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622304cb-0cbb-433f-98ce-8f39c05b9f29 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:55.552521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8849d-ef56-49eb-9de9-71f0f6bd3bde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:55.553183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8849d-ef56-49eb-9de9-71f0f6bd3bde service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:55.553512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8849d-ef56-49eb-9de9-71f0f6bd3bde service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:55.553810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8849d-ef56-49eb-9de9-71f0f6bd3bde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:55.564858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8849d-ef56-49eb-9de9-71f0f6bd3bde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:55.566416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8849d-ef56-49eb-9de9-71f0f6bd3bde service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:55.590938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1108/2205] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:56.047886 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83df65-dc6c-446e-b255-ad9ffd7f2570 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:56.049626 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0b83df65-dc6c-446e-b255-ad9ffd7f2570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:56.071773 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83df65-dc6c-446e-b255-ad9ffd7f2570 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:56.079784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:56.086217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:56.086217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:56.086217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] PATH_INFO: `/v3/projects/362da7f0f55a49289529a9fea00a42d3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:56.086217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=362da7f0f55a49289529a9fea00a42d3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:56.086217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:56.247049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '362da7f0f55a49289529a9fea00a42d3', 'request_id': 'req-690c45d0-b4ab-45cd-9a01-15de8f5080b2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:56.255487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '362da7f0f55a49289529a9fea00a42d3', 'request_id': 'req-690c45d0-b4ab-45cd-9a01-15de8f5080b2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:56.266702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '362da7f0f55a49289529a9fea00a42d3', 'request_id': 'req-690c45d0-b4ab-45cd-9a01-15de8f5080b2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:56.365388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] The token cache is being invalidate because project 362da7f0f55a49289529a9fea00a42d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:01:56.366071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:56.366771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-690c45d0-b4ab-45cd-9a01-15de8f5080b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:01:56.368411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1098/2206] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:01:56 2022] DELETE /identity/v3/projects/362da7f0f55a49289529a9fea00a42d3 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:56.378113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5f8b7-84d6-4d30-afba-35b607fa61cf None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:56.378436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5f8b7-84d6-4d30-afba-35b607fa61cf None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:56.379116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5f8b7-84d6-4d30-afba-35b607fa61cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:56.411765 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7ce5f8b7-84d6-4d30-afba-35b607fa61cf None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:56.422136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7ce5f8b7-84d6-4d30-afba-35b607fa61cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:56.479676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1109/2207] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:56.487535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8cc26-b200-4989-9e42-c53d16644abc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:56.487824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8cc26-b200-4989-9e42-c53d16644abc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:56.488101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8cc26-b200-4989-9e42-c53d16644abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:56.510758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-efe8cc26-b200-4989-9e42-c53d16644abc None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:56.512075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-efe8cc26-b200-4989-9e42-c53d16644abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:56.524772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1099/2208] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:56.528492 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5f8b7-84d6-4d30-afba-35b607fa61cf None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:56.545595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5f8b7-84d6-4d30-afba-35b607fa61cf None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:56.552936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcdd62-6561-40ee-a82d-4203aa44162e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:56.554082 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcdd62-6561-40ee-a82d-4203aa44162e admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:56.554382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcdd62-6561-40ee-a82d-4203aa44162e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:56.555499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcdd62-6561-40ee-a82d-4203aa44162e admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:56.556867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcdd62-6561-40ee-a82d-4203aa44162e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:01:56.557281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcdd62-6561-40ee-a82d-4203aa44162e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:56.559860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcdd62-6561-40ee-a82d-4203aa44162e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:56.577908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1110/2209] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:01:56.582858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8cc26-b200-4989-9e42-c53d16644abc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:56.597227 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8cc26-b200-4989-9e42-c53d16644abc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:56.609263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c7704-9a50-4771-8679-b212e01be46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:56.610807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c7704-9a50-4771-8679-b212e01be46a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:56.611741 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c7704-9a50-4771-8679-b212e01be46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:56.612131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c7704-9a50-4771-8679-b212e01be46a admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:56.612771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c7704-9a50-4771-8679-b212e01be46a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:56.615732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c7704-9a50-4771-8679-b212e01be46a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:56.752827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1100/2210] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:01:56 2022] POST /identity/v3/projects => generated 383 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:56.757178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcdd62-6561-40ee-a82d-4203aa44162e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:56.769499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcdd62-6561-40ee-a82d-4203aa44162e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:56.895549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf063c-0751-46ef-8956-e232b1d804c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:56.896342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf063c-0751-46ef-8956-e232b1d804c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:56.897350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf063c-0751-46ef-8956-e232b1d804c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:56.897858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf063c-0751-46ef-8956-e232b1d804c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:56.898759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf063c-0751-46ef-8956-e232b1d804c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:56.902081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf063c-0751-46ef-8956-e232b1d804c6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:57.056580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1111/2211] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:01:56 2022] POST /identity/v3/users => generated 292 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:01:57.057647 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c7704-9a50-4771-8679-b212e01be46a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:57.085696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c7704-9a50-4771-8679-b212e01be46a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:57.107550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf96e3b-989d-4e11-a447-274143b7bacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:57.112291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf96e3b-989d-4e11-a447-274143b7bacc admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:57.112291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf96e3b-989d-4e11-a447-274143b7bacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:57.112291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf96e3b-989d-4e11-a447-274143b7bacc admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:57.112291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf96e3b-989d-4e11-a447-274143b7bacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:57.112800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf96e3b-989d-4e11-a447-274143b7bacc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:57.124316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1101/2212] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:57.128934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf063c-0751-46ef-8956-e232b1d804c6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:57.140357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf063c-0751-46ef-8956-e232b1d804c6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:57.145916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd4e09-7409-4583-ab40-a72625f648f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:57.146672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd4e09-7409-4583-ab40-a72625f648f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:57.147208 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd4e09-7409-4583-ab40-a72625f648f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:57.147555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd4e09-7409-4583-ab40-a72625f648f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:57.148315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd4e09-7409-4583-ab40-a72625f648f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:57.150020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd4e09-7409-4583-ab40-a72625f648f0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:57.164379 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1112/2213] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:01:57.170126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf96e3b-989d-4e11-a447-274143b7bacc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:57.181110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf96e3b-989d-4e11-a447-274143b7bacc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:57.186506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749b134-00d1-4573-adc4-916e93668e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:57.187271 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749b134-00d1-4573-adc4-916e93668e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:57.188580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749b134-00d1-4573-adc4-916e93668e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:57.188877 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749b134-00d1-4573-adc4-916e93668e84 admin admin] PATH_INFO: `/v3/projects/dd2a37c402a64171aae8a8c90b89ce33/users/acd4018e26564721a8c16db0e9d2e0aa/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:57.212557 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749b134-00d1-4573-adc4-916e93668e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd2a37c402a64171aae8a8c90b89ce33, user_id=acd4018e26564721a8c16db0e9d2e0aa, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:57.215826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749b134-00d1-4573-adc4-916e93668e84 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:57.309856 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1102/2214] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:01:57 2022] PUT /identity/v3/projects/dd2a37c402a64171aae8a8c90b89ce33/users/acd4018e26564721a8c16db0e9d2e0aa/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:01:57.315289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7cc24-bd94-43f8-b1ca-5349a4cbc142 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:57.315574 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7cc24-bd94-43f8-b1ca-5349a4cbc142 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:57.315843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7cc24-bd94-43f8-b1ca-5349a4cbc142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:57.358192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d4e7cc24-bd94-43f8-b1ca-5349a4cbc142 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:57.360062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d4e7cc24-bd94-43f8-b1ca-5349a4cbc142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:57.406866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:57.451636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1113/2215] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:57.479429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749b134-00d1-4573-adc4-916e93668e84 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:57.481497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a749b134-00d1-4573-adc4-916e93668e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:57.535274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:01:57.559592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749b134-00d1-4573-adc4-916e93668e84 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:57.564379 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaef07-fb90-4b04-99ab-dd925395a51f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:57.565002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaef07-fb90-4b04-99ab-dd925395a51f service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:57.565436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaef07-fb90-4b04-99ab-dd925395a51f service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:57.565708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaef07-fb90-4b04-99ab-dd925395a51f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:57.576546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aaef07-fb90-4b04-99ab-dd925395a51f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:57.577922 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aaef07-fb90-4b04-99ab-dd925395a51f service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:57.592191 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1103/2216] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:01:59.223639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a48923-65b8-456b-8683-e67376cf09d2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:59.224178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a48923-65b8-456b-8683-e67376cf09d2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:59.224178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a48923-65b8-456b-8683-e67376cf09d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:59.267719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-21a48923-65b8-456b-8683-e67376cf09d2 None None] MFA Rules not processed for user `acd4018e26564721a8c16db0e9d2e0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:01:59.268869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-21a48923-65b8-456b-8683-e67376cf09d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:59.361475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1114/2217] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:01:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:01:59.373862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaef07-fb90-4b04-99ab-dd925395a51f service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:01:59.376121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d2aaef07-fb90-4b04-99ab-dd925395a51f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:01:59.448194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaef07-fb90-4b04-99ab-dd925395a51f service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:01:59.452215 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e96a8-a437-4276-b539-504eb9afe1bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:01:59.452835 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e96a8-a437-4276-b539-504eb9afe1bf service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:01:59.453743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e96a8-a437-4276-b539-504eb9afe1bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:01:59.454021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e96a8-a437-4276-b539-504eb9afe1bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:01:59.463358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e96a8-a437-4276-b539-504eb9afe1bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:01:59.464991 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e96a8-a437-4276-b539-504eb9afe1bf service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:01:59.474126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1104/2218] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:01:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:02:01.266733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbbe8d-5855-4e82-950b-ca0a7f6f51a2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:01.267211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbbe8d-5855-4e82-950b-ca0a7f6f51a2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:01.267305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbbe8d-5855-4e82-950b-ca0a7f6f51a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:01.286755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-36fbbe8d-5855-4e82-950b-ca0a7f6f51a2 None None] MFA Rules not processed for user `acd4018e26564721a8c16db0e9d2e0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:02:01.287956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-36fbbe8d-5855-4e82-950b-ca0a7f6f51a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:01.295037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1115/2219] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:02:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:02:09.470148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=36 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:02:11.280436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=44 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:02:14.994910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e96a8-a437-4276-b539-504eb9afe1bf service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:14.996461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-886e96a8-a437-4276-b539-504eb9afe1bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:15.012437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e96a8-a437-4276-b539-504eb9afe1bf service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:15.017967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:15.018675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:15.018953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:15.019223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] PATH_INFO: `/v3/users/b44627b2e6ad473da289e0f954647155` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:15.020400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b44627b2e6ad473da289e0f954647155)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:15.022364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:15.211841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b44627b2e6ad473da289e0f954647155'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:15.212517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b44627b2e6ad473da289e0f954647155'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:15.213792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b44627b2e6ad473da289e0f954647155'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:15.295874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b44627b2e6ad473da289e0f954647155'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:15.313386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b44627b2e6ad473da289e0f954647155'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:15.327125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1105/2220] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:02:14 2022] DELETE /identity/v3/users/b44627b2e6ad473da289e0f954647155 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:02:15.332442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5560e2-4d60-4075-82e6-358a3a023359 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:15.333048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5560e2-4d60-4075-82e6-358a3a023359 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:15.333048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5560e2-4d60-4075-82e6-358a3a023359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:15.352357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5d5560e2-4d60-4075-82e6-358a3a023359 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:02:15.353859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5d5560e2-4d60-4075-82e6-358a3a023359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:15.418658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1116/2221] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:02:15.428589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:15.430906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:15.495425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df1835-8819-4937-9501-dcb861283f8c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:15.499664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86644d94-7517-4270-a001-5a7eb1f9d955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:15.502297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86644d94-7517-4270-a001-5a7eb1f9d955 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:15.502297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86644d94-7517-4270-a001-5a7eb1f9d955 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:15.502297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86644d94-7517-4270-a001-5a7eb1f9d955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:15.517723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86644d94-7517-4270-a001-5a7eb1f9d955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:15.519854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86644d94-7517-4270-a001-5a7eb1f9d955 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:15.539244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1106/2222] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:02:15.977593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5560e2-4d60-4075-82e6-358a3a023359 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:15.979041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5d5560e2-4d60-4075-82e6-358a3a023359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:15.991631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5560e2-4d60-4075-82e6-358a3a023359 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:15.996481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:15.997177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:15.997457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:15.997727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] PATH_INFO: `/v3/projects/ec7b9cb55fc04b5c8a37774bbf8edc68` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:15.998855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec7b9cb55fc04b5c8a37774bbf8edc68)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:16.001855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:16.107657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec7b9cb55fc04b5c8a37774bbf8edc68', 'request_id': 'req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:16.113949 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec7b9cb55fc04b5c8a37774bbf8edc68', 'request_id': 'req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:16.120324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec7b9cb55fc04b5c8a37774bbf8edc68', 'request_id': 'req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:16.221469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] The token cache is being invalidate because project ec7b9cb55fc04b5c8a37774bbf8edc68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:02:16.221703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:16.222337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-98af9b70-7fb3-4e1d-8971-6e2bae9fd316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:16.223831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1117/2223] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:02:15 2022] DELETE /identity/v3/projects/ec7b9cb55fc04b5c8a37774bbf8edc68 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:02:16.228643 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad947a3-4621-4bc6-96c3-5ebf7e60b723 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:16.229131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad947a3-4621-4bc6-96c3-5ebf7e60b723 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:16.229225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad947a3-4621-4bc6-96c3-5ebf7e60b723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:16.249478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fad947a3-4621-4bc6-96c3-5ebf7e60b723 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:02:16.251492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fad947a3-4621-4bc6-96c3-5ebf7e60b723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:16.309362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1107/2224] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:02:16.318621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d13bf9-892c-4cee-978f-f49d565cb6cb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:16.318919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d13bf9-892c-4cee-978f-f49d565cb6cb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:16.319179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d13bf9-892c-4cee-978f-f49d565cb6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:16.339651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-45d13bf9-892c-4cee-978f-f49d565cb6cb None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:02:16.340877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-45d13bf9-892c-4cee-978f-f49d565cb6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:16.348915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1118/2225] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:02:16.353065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad947a3-4621-4bc6-96c3-5ebf7e60b723 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:16.363178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad947a3-4621-4bc6-96c3-5ebf7e60b723 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:16.367730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041b7e7-583c-4728-a476-54a78923a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:16.368321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041b7e7-583c-4728-a476-54a78923a4ac admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:16.368602 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041b7e7-583c-4728-a476-54a78923a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:16.368875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041b7e7-583c-4728-a476-54a78923a4ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:16.369562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041b7e7-583c-4728-a476-54a78923a4ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:02:16.369865 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041b7e7-583c-4728-a476-54a78923a4ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:16.371435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041b7e7-583c-4728-a476-54a78923a4ac admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:16.383363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1108/2226] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:02:16.387705 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d13bf9-892c-4cee-978f-f49d565cb6cb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:16.396877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d13bf9-892c-4cee-978f-f49d565cb6cb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:16.401571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247e82d-0614-4c08-94c8-cb9bf87165a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:16.402163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247e82d-0614-4c08-94c8-cb9bf87165a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:16.402462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247e82d-0614-4c08-94c8-cb9bf87165a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:16.402719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247e82d-0614-4c08-94c8-cb9bf87165a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:16.403338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247e82d-0614-4c08-94c8-cb9bf87165a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:16.405253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247e82d-0614-4c08-94c8-cb9bf87165a8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:16.492085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1119/2227] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:02:16 2022] POST /identity/v3/projects => generated 397 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:02:16.496157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041b7e7-583c-4728-a476-54a78923a4ac admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:16.502016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:02:16.507330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041b7e7-583c-4728-a476-54a78923a4ac admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:16.562595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d414e4a-f16f-4547-9f3a-7075693523aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:16.563213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414e4a-f16f-4547-9f3a-7075693523aa admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:16.563697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414e4a-f16f-4547-9f3a-7075693523aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:16.564189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414e4a-f16f-4547-9f3a-7075693523aa admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:16.565291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d414e4a-f16f-4547-9f3a-7075693523aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:16.567218 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d414e4a-f16f-4547-9f3a-7075693523aa admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:16.674592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1109/2228] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:02:16 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:02:16.678719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247e82d-0614-4c08-94c8-cb9bf87165a8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:16.688952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247e82d-0614-4c08-94c8-cb9bf87165a8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:16.693519 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40af72-856a-4b42-a053-4fb4cac15074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:16.694110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40af72-856a-4b42-a053-4fb4cac15074 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:16.694384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40af72-856a-4b42-a053-4fb4cac15074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:16.694651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40af72-856a-4b42-a053-4fb4cac15074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:16.695297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40af72-856a-4b42-a053-4fb4cac15074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:16.696919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40af72-856a-4b42-a053-4fb4cac15074 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:16.707322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1120/2229] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:02:16.715478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d414e4a-f16f-4547-9f3a-7075693523aa admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:16.724093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d414e4a-f16f-4547-9f3a-7075693523aa admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:16.729186 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5012168-a90c-46a6-9696-d02ef0d5f28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:16.729799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5012168-a90c-46a6-9696-d02ef0d5f28c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:16.730080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5012168-a90c-46a6-9696-d02ef0d5f28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:16.730351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5012168-a90c-46a6-9696-d02ef0d5f28c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:16.730995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5012168-a90c-46a6-9696-d02ef0d5f28c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:16.732578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5012168-a90c-46a6-9696-d02ef0d5f28c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:16.747008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1110/2230] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:02:16.748148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40af72-856a-4b42-a053-4fb4cac15074 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:16.758471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40af72-856a-4b42-a053-4fb4cac15074 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:16.763361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856c17b-7487-4677-b369-49ebe842ef44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:16.764125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856c17b-7487-4677-b369-49ebe842ef44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:16.764504 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856c17b-7487-4677-b369-49ebe842ef44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:16.764895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856c17b-7487-4677-b369-49ebe842ef44 admin admin] PATH_INFO: `/v3/projects/80cb1c07d2c0454e96c28289ddbd0490/users/4017445a3a6f402ebe9f46168dd9f32e/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:16.782255 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856c17b-7487-4677-b369-49ebe842ef44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cb1c07d2c0454e96c28289ddbd0490, user_id=4017445a3a6f402ebe9f46168dd9f32e, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:16.785624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856c17b-7487-4677-b369-49ebe842ef44 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:16.873776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1121/2231] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:02:16 2022] PUT /identity/v3/projects/80cb1c07d2c0454e96c28289ddbd0490/users/4017445a3a6f402ebe9f46168dd9f32e/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:02:16.885331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6a6cb-1eb4-4e84-a99d-684984eaa38b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:16.885726 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6a6cb-1eb4-4e84-a99d-684984eaa38b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:16.885901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6a6cb-1eb4-4e84-a99d-684984eaa38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:16.915131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-26e6a6cb-1eb4-4e84-a99d-684984eaa38b None None] MFA Rules not processed for user `4017445a3a6f402ebe9f46168dd9f32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:02:16.916351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-26e6a6cb-1eb4-4e84-a99d-684984eaa38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:16.981757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1111/2232] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:02:16 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:02:16.993086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856c17b-7487-4677-b369-49ebe842ef44 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:16.996432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e856c17b-7487-4677-b369-49ebe842ef44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:17.017097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:02:17.052362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856c17b-7487-4677-b369-49ebe842ef44 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:17.056198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148ef80-e04d-448b-ad22-bfd6571bc2fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:17.056769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148ef80-e04d-448b-ad22-bfd6571bc2fd service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:17.057073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148ef80-e04d-448b-ad22-bfd6571bc2fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:17.057356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148ef80-e04d-448b-ad22-bfd6571bc2fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:17.066353 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148ef80-e04d-448b-ad22-bfd6571bc2fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:17.067820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148ef80-e04d-448b-ad22-bfd6571bc2fd service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:17.076806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1122/2233] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:02:16 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:02:26.041341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6a6cb-1eb4-4e84-a99d-684984eaa38b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:26.103551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6a6cb-1eb4-4e84-a99d-684984eaa38b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:26.108083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:26.108715 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:26.108999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:26.109898 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] PATH_INFO: `/v3/users/4017445a3a6f402ebe9f46168dd9f32e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:26.111035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4017445a3a6f402ebe9f46168dd9f32e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:26.112938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:26.126392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:02:26.310572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4017445a3a6f402ebe9f46168dd9f32e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:26.311235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4017445a3a6f402ebe9f46168dd9f32e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:26.311854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4017445a3a6f402ebe9f46168dd9f32e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:26.404432 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4017445a3a6f402ebe9f46168dd9f32e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:26.418322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4017445a3a6f402ebe9f46168dd9f32e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:26.431496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1112/2234] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:02:26 2022] DELETE /identity/v3/users/4017445a3a6f402ebe9f46168dd9f32e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:02:26.435976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082e585-4b4b-4afd-be30-ca61c9a5be61 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:26.436462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082e585-4b4b-4afd-be30-ca61c9a5be61 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:26.436567 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082e585-4b4b-4afd-be30-ca61c9a5be61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:26.456113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5082e585-4b4b-4afd-be30-ca61c9a5be61 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:02:26.457814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5082e585-4b4b-4afd-be30-ca61c9a5be61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:26.519112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1123/2235] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:02:26.528931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:26.535584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:26.591866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a309e7-1420-4330-a296-1d91710d8363 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:26.596211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68715fb4-3c5f-4b27-afb9-6a3544caac97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:26.596829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68715fb4-3c5f-4b27-afb9-6a3544caac97 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:26.597218 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68715fb4-3c5f-4b27-afb9-6a3544caac97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:26.597480 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68715fb4-3c5f-4b27-afb9-6a3544caac97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:26.606750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68715fb4-3c5f-4b27-afb9-6a3544caac97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:26.610163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68715fb4-3c5f-4b27-afb9-6a3544caac97 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:26.618341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1113/2236] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:02:27.034854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082e585-4b4b-4afd-be30-ca61c9a5be61 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:27.036513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5082e585-4b4b-4afd-be30-ca61c9a5be61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:27.056734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082e585-4b4b-4afd-be30-ca61c9a5be61 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:27.061672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:27.062822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:27.063126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:27.063440 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] PATH_INFO: `/v3/projects/80cb1c07d2c0454e96c28289ddbd0490` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:27.064598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80cb1c07d2c0454e96c28289ddbd0490)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:27.066536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:27.204104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80cb1c07d2c0454e96c28289ddbd0490', 'request_id': 'req-9298c166-ea9a-496a-b387-031ad71c65b1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:27.209538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80cb1c07d2c0454e96c28289ddbd0490', 'request_id': 'req-9298c166-ea9a-496a-b387-031ad71c65b1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:27.215987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80cb1c07d2c0454e96c28289ddbd0490', 'request_id': 'req-9298c166-ea9a-496a-b387-031ad71c65b1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:27.289651 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] The token cache is being invalidate because project 80cb1c07d2c0454e96c28289ddbd0490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:02:27.289863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:27.290469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9298c166-ea9a-496a-b387-031ad71c65b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:02:27.291955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1124/2237] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:02:27 2022] DELETE /identity/v3/projects/80cb1c07d2c0454e96c28289ddbd0490 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:02:27.297385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c604e-7681-4d8a-bbba-7a44ba7888f3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:27.297861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c604e-7681-4d8a-bbba-7a44ba7888f3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:27.297964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c604e-7681-4d8a-bbba-7a44ba7888f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:27.318325 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-300c604e-7681-4d8a-bbba-7a44ba7888f3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:02:27.319549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-300c604e-7681-4d8a-bbba-7a44ba7888f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:27.378728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1114/2238] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:02:27.386597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd67df-02fc-4cd1-b5d6-e57bca7f5147 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:27.388246 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd67df-02fc-4cd1-b5d6-e57bca7f5147 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:27.388246 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd67df-02fc-4cd1-b5d6-e57bca7f5147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:27.407268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-51dd67df-02fc-4cd1-b5d6-e57bca7f5147 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:02:27.411253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-51dd67df-02fc-4cd1-b5d6-e57bca7f5147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:27.416427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1125/2239] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:02:27.420784 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c604e-7681-4d8a-bbba-7a44ba7888f3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:27.431600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c604e-7681-4d8a-bbba-7a44ba7888f3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:27.436531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ecec4-01fa-4034-9e07-4d97ba1045a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:27.437220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ecec4-01fa-4034-9e07-4d97ba1045a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:27.437504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ecec4-01fa-4034-9e07-4d97ba1045a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:27.437780 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ecec4-01fa-4034-9e07-4d97ba1045a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:27.439360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ecec4-01fa-4034-9e07-4d97ba1045a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:02:27.439744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ecec4-01fa-4034-9e07-4d97ba1045a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:27.441390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ecec4-01fa-4034-9e07-4d97ba1045a6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:27.453183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1115/2240] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:02:27.457711 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd67df-02fc-4cd1-b5d6-e57bca7f5147 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:27.467550 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd67df-02fc-4cd1-b5d6-e57bca7f5147 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:27.472560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058777f-d1c4-4697-8b53-f9c369040b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:27.473201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058777f-d1c4-4697-8b53-f9c369040b3e admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:27.473511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058777f-d1c4-4697-8b53-f9c369040b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:27.473786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058777f-d1c4-4697-8b53-f9c369040b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:27.474441 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058777f-d1c4-4697-8b53-f9c369040b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:27.476386 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058777f-d1c4-4697-8b53-f9c369040b3e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:27.567685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1126/2241] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:02:27 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:02:27.571891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ecec4-01fa-4034-9e07-4d97ba1045a6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:27.582653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ecec4-01fa-4034-9e07-4d97ba1045a6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:27.602466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:02:27.643174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1a858-1747-469a-9c6b-069a529f4dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:27.643780 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1a858-1747-469a-9c6b-069a529f4dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:27.644053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1a858-1747-469a-9c6b-069a529f4dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:27.644373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1a858-1747-469a-9c6b-069a529f4dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:27.644951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1a858-1747-469a-9c6b-069a529f4dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:27.646900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1a858-1747-469a-9c6b-069a529f4dd9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:27.737502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1116/2242] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:02:27 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:02:27.741619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058777f-d1c4-4697-8b53-f9c369040b3e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:27.752029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058777f-d1c4-4697-8b53-f9c369040b3e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:27.756584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbe7f8-cf34-407c-aafc-2b2dd81c92e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:27.757239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbe7f8-cf34-407c-aafc-2b2dd81c92e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:27.757560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbe7f8-cf34-407c-aafc-2b2dd81c92e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:27.757830 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbe7f8-cf34-407c-aafc-2b2dd81c92e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:27.758474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbe7f8-cf34-407c-aafc-2b2dd81c92e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:27.760061 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbe7f8-cf34-407c-aafc-2b2dd81c92e0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:27.771780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1127/2243] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:02:27.774779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1a858-1747-469a-9c6b-069a529f4dd9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:27.788222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1a858-1747-469a-9c6b-069a529f4dd9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:27.796318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13008045-d880-4e63-9cde-c8334d4f7b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:27.797453 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13008045-d880-4e63-9cde-c8334d4f7b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:27.798546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13008045-d880-4e63-9cde-c8334d4f7b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:27.799023 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13008045-d880-4e63-9cde-c8334d4f7b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:27.800165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13008045-d880-4e63-9cde-c8334d4f7b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:27.803320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13008045-d880-4e63-9cde-c8334d4f7b0b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:27.815151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1117/2244] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:02:27.819513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbe7f8-cf34-407c-aafc-2b2dd81c92e0 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:27.831192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbe7f8-cf34-407c-aafc-2b2dd81c92e0 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:27.835920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb086c4-a62b-4e1d-a2ef-d33bf85d9a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:27.836580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb086c4-a62b-4e1d-a2ef-d33bf85d9a80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:27.836860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb086c4-a62b-4e1d-a2ef-d33bf85d9a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:27.837173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb086c4-a62b-4e1d-a2ef-d33bf85d9a80 admin admin] PATH_INFO: `/v3/projects/ea7b166d3468425ab368e369b4dd9989/users/ee81d8d9cf994df68a15afe5cb9673c4/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:27.839754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:02:27.852945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb086c4-a62b-4e1d-a2ef-d33bf85d9a80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7b166d3468425ab368e369b4dd9989, user_id=ee81d8d9cf994df68a15afe5cb9673c4, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:27.855641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb086c4-a62b-4e1d-a2ef-d33bf85d9a80 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:27.925480 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1128/2245] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:02:27 2022] PUT /identity/v3/projects/ea7b166d3468425ab368e369b4dd9989/users/ee81d8d9cf994df68a15afe5cb9673c4/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:02:27.936474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12538a-fa25-4b5a-a7f8-f49b91346741 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:27.936786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12538a-fa25-4b5a-a7f8-f49b91346741 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:27.937095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12538a-fa25-4b5a-a7f8-f49b91346741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:27.956631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fa12538a-fa25-4b5a-a7f8-f49b91346741 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:02:27.957882 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fa12538a-fa25-4b5a-a7f8-f49b91346741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:28.017833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1118/2246] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:02:28.027531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb086c4-a62b-4e1d-a2ef-d33bf85d9a80 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:28.029365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5bb086c4-a62b-4e1d-a2ef-d33bf85d9a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:28.088045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb086c4-a62b-4e1d-a2ef-d33bf85d9a80 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:28.092104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1dcaa-f95e-4069-9eae-bbebca60bfdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:28.092719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1dcaa-f95e-4069-9eae-bbebca60bfdd service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:28.093001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1dcaa-f95e-4069-9eae-bbebca60bfdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:28.093301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1dcaa-f95e-4069-9eae-bbebca60bfdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:28.103650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1dcaa-f95e-4069-9eae-bbebca60bfdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:28.105109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1dcaa-f95e-4069-9eae-bbebca60bfdd service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:28.116823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1129/2247] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:02:33.191248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56253a-2afc-495c-85f5-2df4069ff4ed None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:33.191746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56253a-2afc-495c-85f5-2df4069ff4ed None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:33.191827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56253a-2afc-495c-85f5-2df4069ff4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:33.225485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ca56253a-2afc-495c-85f5-2df4069ff4ed None None] MFA Rules not processed for user `ee81d8d9cf994df68a15afe5cb9673c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:02:33.226721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ca56253a-2afc-495c-85f5-2df4069ff4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:33.300672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1119/2248] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:02:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:02:33.305287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1dcaa-f95e-4069-9eae-bbebca60bfdd service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:33.315534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1dcaa-f95e-4069-9eae-bbebca60bfdd service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:33.320591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39047d0e-4cb3-4a2d-b5b6-d2df66ea7963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:33.321261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39047d0e-4cb3-4a2d-b5b6-d2df66ea7963 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:33.321544 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39047d0e-4cb3-4a2d-b5b6-d2df66ea7963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:33.321815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39047d0e-4cb3-4a2d-b5b6-d2df66ea7963 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:33.322453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39047d0e-4cb3-4a2d-b5b6-d2df66ea7963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:33.324365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39047d0e-4cb3-4a2d-b5b6-d2df66ea7963 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:33.429254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1130/2249] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:02:33 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:02:33.435583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56253a-2afc-495c-85f5-2df4069ff4ed None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:33.450087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56253a-2afc-495c-85f5-2df4069ff4ed None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:33.479159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:02:33.511595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c39af-c61f-484b-946c-beabc41bff82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:33.512208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c39af-c61f-484b-946c-beabc41bff82 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:33.512547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c39af-c61f-484b-946c-beabc41bff82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:33.512757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c39af-c61f-484b-946c-beabc41bff82 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:33.513443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c39af-c61f-484b-946c-beabc41bff82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:33.515364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c39af-c61f-484b-946c-beabc41bff82 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:33.603208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1120/2250] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:02:33 2022] POST /identity/v3/users => generated 296 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:02:33.605862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39047d0e-4cb3-4a2d-b5b6-d2df66ea7963 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:33.617095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39047d0e-4cb3-4a2d-b5b6-d2df66ea7963 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:33.622490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cf285-21f7-4e3e-ae91-0d306642b935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:33.623110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf285-21f7-4e3e-ae91-0d306642b935 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:33.623424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf285-21f7-4e3e-ae91-0d306642b935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:33.623670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf285-21f7-4e3e-ae91-0d306642b935 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:33.624322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cf285-21f7-4e3e-ae91-0d306642b935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:33.626039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cf285-21f7-4e3e-ae91-0d306642b935 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:33.640562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1131/2251] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:02:33.645534 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c39af-c61f-484b-946c-beabc41bff82 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:33.657195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c39af-c61f-484b-946c-beabc41bff82 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:33.668366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dbb10-47f0-4180-8c34-b4f758354d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:33.669133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dbb10-47f0-4180-8c34-b4f758354d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:33.669419 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dbb10-47f0-4180-8c34-b4f758354d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:33.669699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dbb10-47f0-4180-8c34-b4f758354d41 admin admin] PATH_INFO: `/v3/projects/dd5c1df78c1d4190bda02186eedb3ff9/users/df3f4b89f25c4d06a4dfeb65c4fb52e7/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:33.686409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dbb10-47f0-4180-8c34-b4f758354d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd5c1df78c1d4190bda02186eedb3ff9, user_id=df3f4b89f25c4d06a4dfeb65c4fb52e7, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:33.689312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dbb10-47f0-4180-8c34-b4f758354d41 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:33.759155 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1121/2252] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:02:33 2022] PUT /identity/v3/projects/dd5c1df78c1d4190bda02186eedb3ff9/users/df3f4b89f25c4d06a4dfeb65c4fb52e7/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:02:33.769554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cf285-21f7-4e3e-ae91-0d306642b935 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:33.773669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cf285-21f7-4e3e-ae91-0d306642b935 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:33.831934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a5e5b-ee76-4a58-9561-28ec142890a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:33.832549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a5e5b-ee76-4a58-9561-28ec142890a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:33.835251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a5e5b-ee76-4a58-9561-28ec142890a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:33.835251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a5e5b-ee76-4a58-9561-28ec142890a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:33.835251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a5e5b-ee76-4a58-9561-28ec142890a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:33.835520 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a5e5b-ee76-4a58-9561-28ec142890a1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:33.846784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1132/2253] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:02:33.850953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dbb10-47f0-4180-8c34-b4f758354d41 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:33.862916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dbb10-47f0-4180-8c34-b4f758354d41 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:33.868527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb7f75-1e94-4bb8-a3ca-4c0e95bfb949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:33.869388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7f75-1e94-4bb8-a3ca-4c0e95bfb949 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:33.869592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7f75-1e94-4bb8-a3ca-4c0e95bfb949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:33.869899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7f75-1e94-4bb8-a3ca-4c0e95bfb949 admin admin] PATH_INFO: `/v3/projects/dd5c1df78c1d4190bda02186eedb3ff9/users/df3f4b89f25c4d06a4dfeb65c4fb52e7/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:33.872043 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb7f75-1e94-4bb8-a3ca-4c0e95bfb949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd5c1df78c1d4190bda02186eedb3ff9, user_id=df3f4b89f25c4d06a4dfeb65c4fb52e7, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:33.874736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb7f75-1e94-4bb8-a3ca-4c0e95bfb949 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:33.889105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1122/2254] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:02:33 2022] PUT /identity/v3/projects/dd5c1df78c1d4190bda02186eedb3ff9/users/df3f4b89f25c4d06a4dfeb65c4fb52e7/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:02:39.266287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cb3c1-d26d-4de8-9a0f-c3b8fc1d9312 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:39.266783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cb3c1-d26d-4de8-9a0f-c3b8fc1d9312 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:39.266866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484cb3c1-d26d-4de8-9a0f-c3b8fc1d9312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:39.297055 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-484cb3c1-d26d-4de8-9a0f-c3b8fc1d9312 None None] MFA Rules not processed for user `df3f4b89f25c4d06a4dfeb65c4fb52e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:02:39.298327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-484cb3c1-d26d-4de8-9a0f-c3b8fc1d9312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:39.340241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:02:39.372096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1133/2255] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:02:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:02:39.393263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb7f75-1e94-4bb8-a3ca-4c0e95bfb949 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:02:39.401517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bbcb7f75-1e94-4bb8-a3ca-4c0e95bfb949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:02:39.498172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb7f75-1e94-4bb8-a3ca-4c0e95bfb949 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:02:39.504213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba058701-d649-4fb4-9001-6061c7d12dce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:02:39.504742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba058701-d649-4fb4-9001-6061c7d12dce service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:02:39.505311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba058701-d649-4fb4-9001-6061c7d12dce service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:02:39.505311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba058701-d649-4fb4-9001-6061c7d12dce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:02:39.574659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba058701-d649-4fb4-9001-6061c7d12dce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:02:39.576417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba058701-d649-4fb4-9001-6061c7d12dce service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:02:39.587300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1123/2256] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:02:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:02:49.552903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=12 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:02:49.582667 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:03:05.154373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe67f5-6e15-4461-a54c-19b2a5113eba None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:05.154919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe67f5-6e15-4461-a54c-19b2a5113eba None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:05.154919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe67f5-6e15-4461-a54c-19b2a5113eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:05.173657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-16fe67f5-6e15-4461-a54c-19b2a5113eba None None] MFA Rules not processed for user `ee81d8d9cf994df68a15afe5cb9673c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:03:05.174895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-16fe67f5-6e15-4461-a54c-19b2a5113eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:05.182395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1134/2257] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:03:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:03:08.107203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba058701-d649-4fb4-9001-6061c7d12dce service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:03:08.109399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ba058701-d649-4fb4-9001-6061c7d12dce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:08.208681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba058701-d649-4fb4-9001-6061c7d12dce service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:03:08.216543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e68213-4a0a-4414-a3a1-17d8a1b2da55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:03:08.217210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e68213-4a0a-4414-a3a1-17d8a1b2da55 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:08.217515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e68213-4a0a-4414-a3a1-17d8a1b2da55 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:08.220176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e68213-4a0a-4414-a3a1-17d8a1b2da55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:08.220176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-72e68213-4a0a-4414-a3a1-17d8a1b2da55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:08.232545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e68213-4a0a-4414-a3a1-17d8a1b2da55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:03:08.234209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e68213-4a0a-4414-a3a1-17d8a1b2da55 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:03:08.245314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1124/2258] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:03:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:03:13.752429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ba1ce-36b8-4956-8939-4ce876aa14a6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:13.752923 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ba1ce-36b8-4956-8939-4ce876aa14a6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:13.753043 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ba1ce-36b8-4956-8939-4ce876aa14a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:13.772520 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-351ba1ce-36b8-4956-8939-4ce876aa14a6 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:03:13.773847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-351ba1ce-36b8-4956-8939-4ce876aa14a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:13.887051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1135/2259] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 10:03:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:03:13.948412 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e68213-4a0a-4414-a3a1-17d8a1b2da55 service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:03:13.949984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-72e68213-4a0a-4414-a3a1-17d8a1b2da55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:14.076523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e68213-4a0a-4414-a3a1-17d8a1b2da55 service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:03:14.099447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0245271-6510-4d76-9eb8-0d3118609478 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:03:14.100494 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0245271-6510-4d76-9eb8-0d3118609478 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:14.100791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0245271-6510-4d76-9eb8-0d3118609478 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:14.101156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0245271-6510-4d76-9eb8-0d3118609478 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:14.119257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0245271-6510-4d76-9eb8-0d3118609478 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:03:14.121105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0245271-6510-4d76-9eb8-0d3118609478 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:03:14.132060 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1125/2260] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:03:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:03:23.874391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=28 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:03:24.126457 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=64 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:03:34.966932 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c04796-1a8b-4551-9cc2-6adc8b847690 None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:34.967485 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c04796-1a8b-4551-9cc2-6adc8b847690 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:34.967593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c04796-1a8b-4551-9cc2-6adc8b847690 None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:34.968953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1136/2261] 10.222.0.17 () {54 vars in 921 bytes} [Thu Aug 11 10:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:03:34.975576 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9d130-f22a-411b-bf5a-fd76ad254c14 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:34.976074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9d130-f22a-411b-bf5a-fd76ad254c14 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:34.976478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9d130-f22a-411b-bf5a-fd76ad254c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:35.001636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cdd9d130-f22a-411b-bf5a-fd76ad254c14 None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:03:35.002959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cdd9d130-f22a-411b-bf5a-fd76ad254c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:35.012965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1126/2262] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Aug 11 10:03:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:03:35.024848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c04796-1a8b-4551-9cc2-6adc8b847690 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:03:35.026443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e2c04796-1a8b-4551-9cc2-6adc8b847690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:35.091895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c04796-1a8b-4551-9cc2-6adc8b847690 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:03:35.095477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fc3ac-b4d1-4977-ad2d-90ce9dd12b77 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83448e4c6704f7c8771132bd67eaa1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:03:35.096086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fc3ac-b4d1-4977-ad2d-90ce9dd12b77 service glance] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:35.096360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fc3ac-b4d1-4977-ad2d-90ce9dd12b77 service glance] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:35.096628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fc3ac-b4d1-4977-ad2d-90ce9dd12b77 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:35.106915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fc3ac-b4d1-4977-ad2d-90ce9dd12b77 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:03:35.108338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fc3ac-b4d1-4977-ad2d-90ce9dd12b77 service glance] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:03:35.120600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1137/2263] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:03:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:03:35.308493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec07d9-85c7-40d6-84ec-133c9ae4bca7 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:35.308864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec07d9-85c7-40d6-84ec-133c9ae4bca7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:35.309250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec07d9-85c7-40d6-84ec-133c9ae4bca7 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:35.310619 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1127/2264] 10.222.0.17 () {54 vars in 921 bytes} [Thu Aug 11 10:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:03:35.315690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d751016-2ac9-40a8-98bb-b4ac5bf2bc00 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:35.320196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d751016-2ac9-40a8-98bb-b4ac5bf2bc00 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:35.320196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d751016-2ac9-40a8-98bb-b4ac5bf2bc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:35.336133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1d751016-2ac9-40a8-98bb-b4ac5bf2bc00 None None] MFA Rules not processed for user `4fc6c3d1ce8645808c22d1b59b15a100`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:03:35.337454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1d751016-2ac9-40a8-98bb-b4ac5bf2bc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:35.391461 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1138/2265] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Aug 11 10:03:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:03:35.403851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec07d9-85c7-40d6-84ec-133c9ae4bca7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:03:35.405567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-41ec07d9-85c7-40d6-84ec-133c9ae4bca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:35.419822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec07d9-85c7-40d6-84ec-133c9ae4bca7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:03:35.423340 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98889db1-3086-4cd8-a477-b63227c55ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:03:35.423930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98889db1-3086-4cd8-a477-b63227c55ecb service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:35.424211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98889db1-3086-4cd8-a477-b63227c55ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:35.424506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98889db1-3086-4cd8-a477-b63227c55ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:35.432928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98889db1-3086-4cd8-a477-b63227c55ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:03:35.434566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98889db1-3086-4cd8-a477-b63227c55ecb service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:03:35.444125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1128/2266] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:03:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:03:35.527279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a143ee-37a6-46af-9eb8-06c31828659f None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:35.527592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a143ee-37a6-46af-9eb8-06c31828659f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:35.527866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a143ee-37a6-46af-9eb8-06c31828659f None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:35.529247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1139/2267] 10.222.0.17 () {54 vars in 921 bytes} [Thu Aug 11 10:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:03:35.534099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be109fe1-5d67-4d1a-8dc5-c6576f819f4d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:35.534379 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be109fe1-5d67-4d1a-8dc5-c6576f819f4d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:35.534675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be109fe1-5d67-4d1a-8dc5-c6576f819f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:35.554861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-be109fe1-5d67-4d1a-8dc5-c6576f819f4d None None] MFA Rules not processed for user `34fa13a87ebe4b789ca1884e5aeb9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:03:35.556038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-be109fe1-5d67-4d1a-8dc5-c6576f819f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:35.565106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1129/2268] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Aug 11 10:03:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:03:35.575441 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a143ee-37a6-46af-9eb8-06c31828659f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:03:35.586141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a143ee-37a6-46af-9eb8-06c31828659f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:03:35.589645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4103d-b52c-4a30-b05f-e045073d8bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:03:35.590477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4103d-b52c-4a30-b05f-e045073d8bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:35.590758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4103d-b52c-4a30-b05f-e045073d8bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:35.591041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4103d-b52c-4a30-b05f-e045073d8bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:35.601552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4103d-b52c-4a30-b05f-e045073d8bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:03:35.602982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4103d-b52c-4a30-b05f-e045073d8bb9 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:03:35.613347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1140/2269] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:03:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:03:38.072808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be109fe1-5d67-4d1a-8dc5-c6576f819f4d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:03:38.082548 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be109fe1-5d67-4d1a-8dc5-c6576f819f4d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:03:38.087582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe3c9a-c05b-4503-a8e8-0b42aa58bc28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:03:38.088186 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe3c9a-c05b-4503-a8e8-0b42aa58bc28 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:38.088485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe3c9a-c05b-4503-a8e8-0b42aa58bc28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:38.088754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe3c9a-c05b-4503-a8e8-0b42aa58bc28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:38.098878 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe3c9a-c05b-4503-a8e8-0b42aa58bc28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:03:38.100275 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe3c9a-c05b-4503-a8e8-0b42aa58bc28 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:03:38.109486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1130/2270] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:03:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:03:43.492833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afbc97-4aae-4ea2-9214-56354d6069c5 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:43.499261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afbc97-4aae-4ea2-9214-56354d6069c5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:43.499261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afbc97-4aae-4ea2-9214-56354d6069c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:43.512703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-76afbc97-4aae-4ea2-9214-56354d6069c5 None None] MFA Rules not processed for user `acd4018e26564721a8c16db0e9d2e0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:03:43.514025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-76afbc97-4aae-4ea2-9214-56354d6069c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:43.569633 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1141/2271] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:03:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:03:44.620565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a9b9a-eef5-47a1-a8eb-608744f8c352 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:44.621419 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a9b9a-eef5-47a1-a8eb-608744f8c352 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:44.621848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a9b9a-eef5-47a1-a8eb-608744f8c352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:44.642572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-229a9b9a-eef5-47a1-a8eb-608744f8c352 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:03:44.644021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-229a9b9a-eef5-47a1-a8eb-608744f8c352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:03:44.654291 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1131/2272] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 10:03:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:03:44.668253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afbc97-4aae-4ea2-9214-56354d6069c5 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:03:44.681228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afbc97-4aae-4ea2-9214-56354d6069c5 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:03:44.686564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a7b939-a622-4b3f-84e7-7c94f709cd45 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:03:44.687178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7b939-a622-4b3f-84e7-7c94f709cd45 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:03:44.687462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7b939-a622-4b3f-84e7-7c94f709cd45 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:03:44.687744 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a7b939-a622-4b3f-84e7-7c94f709cd45 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:03:44.696518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7b939-a622-4b3f-84e7-7c94f709cd45 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:03:44.698159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a7b939-a622-4b3f-84e7-7c94f709cd45 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:03:44.706842 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1142/2273] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:03:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:03:54.637205 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=21 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:03:54.703108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=86 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:04:05.046611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d484973-1dbe-48cf-8700-0ad9e51edcf3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:04:05.047199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d484973-1dbe-48cf-8700-0ad9e51edcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:04:05.047581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d484973-1dbe-48cf-8700-0ad9e51edcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:04:05.086680 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1d484973-1dbe-48cf-8700-0ad9e51edcf3 None None] MFA Rules not processed for user `c62f05ea732142c1af1c09dd23e7b9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:04:05.088115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1d484973-1dbe-48cf-8700-0ad9e51edcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:04:05.154758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1132/2274] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:04:05.313917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264488d-3272-43bc-b0e8-f3c83ce5e8c6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:04:05.314509 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264488d-3272-43bc-b0e8-f3c83ce5e8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:04:05.314509 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264488d-3272-43bc-b0e8-f3c83ce5e8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:04:05.333708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f264488d-3272-43bc-b0e8-f3c83ce5e8c6 None None] MFA Rules not processed for user `28b31650305943b498f02c882baa0144`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:04:05.334924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f264488d-3272-43bc-b0e8-f3c83ce5e8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:04:05.390104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1143/2275] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:04:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:04:08.579169 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67162c55-ef59-4a4e-b1c9-4e94b375d9ac None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:04:08.579705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67162c55-ef59-4a4e-b1c9-4e94b375d9ac None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:04:08.580221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67162c55-ef59-4a4e-b1c9-4e94b375d9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:04:08.601468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-67162c55-ef59-4a4e-b1c9-4e94b375d9ac None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:04:08.603503 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-67162c55-ef59-4a4e-b1c9-4e94b375d9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:04:08.618941 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1133/2276] 10.222.0.17 () {58 vars in 967 bytes} [Thu Aug 11 10:04:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:04:08.626264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264488d-3272-43bc-b0e8-f3c83ce5e8c6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:04:08.641877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264488d-3272-43bc-b0e8-f3c83ce5e8c6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:04:08.647828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bd03b-eb75-4f81-837b-cf478a425520 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:04:08.648971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bd03b-eb75-4f81-837b-cf478a425520 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:04:08.649324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bd03b-eb75-4f81-837b-cf478a425520 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:04:08.649621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bd03b-eb75-4f81-837b-cf478a425520 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:04:08.660031 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512bd03b-eb75-4f81-837b-cf478a425520 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:04:08.663073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512bd03b-eb75-4f81-837b-cf478a425520 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:04:08.672289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1144/2277] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:04:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:04:18.596135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=28 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:04:18.668445 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=31 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:04:22.079750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c91bbd-153b-4859-a65e-c4de9b5ecf36 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:04:22.080255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c91bbd-153b-4859-a65e-c4de9b5ecf36 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:04:22.080348 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c91bbd-153b-4859-a65e-c4de9b5ecf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:04:22.099463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-66c91bbd-153b-4859-a65e-c4de9b5ecf36 None None] MFA Rules not processed for user `d6c2fc568ee74d35bba01c6ce294edeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:04:22.101672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-66c91bbd-153b-4859-a65e-c4de9b5ecf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:04:22.155542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1134/2278] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:04:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:04:30.846344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bd03b-eb75-4f81-837b-cf478a425520 service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:04:30.856741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bd03b-eb75-4f81-837b-cf478a425520 service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:04:30.860707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42989336-c104-49dc-acbe-e398489a9add service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:04:30.861342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42989336-c104-49dc-acbe-e398489a9add service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:04:30.861623 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42989336-c104-49dc-acbe-e398489a9add service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:04:30.861895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42989336-c104-49dc-acbe-e398489a9add service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:04:30.863618 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-42989336-c104-49dc-acbe-e398489a9add service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:04:30.878005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42989336-c104-49dc-acbe-e398489a9add service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:04:30.879428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42989336-c104-49dc-acbe-e398489a9add service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:04:30.891110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1145/2279] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:04:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:04:32.148666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=25 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:04:33.323126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c91bbd-153b-4859-a65e-c4de9b5ecf36 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:04:33.333445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c91bbd-153b-4859-a65e-c4de9b5ecf36 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:04:33.337192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46f2a5-2994-4a4c-97e3-04af179d7c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:04:33.338261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f2a5-2994-4a4c-97e3-04af179d7c13 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:04:33.338261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f2a5-2994-4a4c-97e3-04af179d7c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:04:33.338456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f2a5-2994-4a4c-97e3-04af179d7c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:04:33.340057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-db46f2a5-2994-4a4c-97e3-04af179d7c13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:04:33.353737 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46f2a5-2994-4a4c-97e3-04af179d7c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:04:33.355140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46f2a5-2994-4a4c-97e3-04af179d7c13 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:04:33.366198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1135/2280] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:04:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:04:40.886456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:04:43.361198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:04:43.954843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42989336-c104-49dc-acbe-e398489a9add service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:04:43.967663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42989336-c104-49dc-acbe-e398489a9add service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:04:43.973197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d65ec-c7bd-4889-9bb7-674e25b0b98f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:04:43.973793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d65ec-c7bd-4889-9bb7-674e25b0b98f service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:04:43.974123 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d65ec-c7bd-4889-9bb7-674e25b0b98f service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:04:43.974635 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d65ec-c7bd-4889-9bb7-674e25b0b98f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:04:43.976394 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-337d65ec-c7bd-4889-9bb7-674e25b0b98f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:04:43.986758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d65ec-c7bd-4889-9bb7-674e25b0b98f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:04:43.988367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d65ec-c7bd-4889-9bb7-674e25b0b98f service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:04:43.997934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1146/2281] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:04:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:04:53.993754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=6 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:00.254595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46f2a5-2994-4a4c-97e3-04af179d7c13 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:00.256539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-db46f2a5-2994-4a4c-97e3-04af179d7c13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:00.314629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46f2a5-2994-4a4c-97e3-04af179d7c13 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:00.318876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18045b13-ed9c-41ad-b282-3032a15115e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:00.319469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18045b13-ed9c-41ad-b282-3032a15115e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:00.319745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18045b13-ed9c-41ad-b282-3032a15115e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:00.320016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18045b13-ed9c-41ad-b282-3032a15115e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:00.321813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-18045b13-ed9c-41ad-b282-3032a15115e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:00.332928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18045b13-ed9c-41ad-b282-3032a15115e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:00.335080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18045b13-ed9c-41ad-b282-3032a15115e8 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:00.344574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1136/2282] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:08.307859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d65ec-c7bd-4889-9bb7-674e25b0b98f service nova] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:08.309566 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-337d65ec-c7bd-4889-9bb7-674e25b0b98f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:08.325247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d65ec-c7bd-4889-9bb7-674e25b0b98f service nova] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:08.330871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916d1fc-b25b-4b78-80c4-c873117193ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:08.331506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916d1fc-b25b-4b78-80c4-c873117193ec service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:08.331790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916d1fc-b25b-4b78-80c4-c873117193ec service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:08.332063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916d1fc-b25b-4b78-80c4-c873117193ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:08.334006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a916d1fc-b25b-4b78-80c4-c873117193ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:08.347473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916d1fc-b25b-4b78-80c4-c873117193ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:08.349146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916d1fc-b25b-4b78-80c4-c873117193ec service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:08.360364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1147/2283] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:10.340002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=26 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:14.794501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18045b13-ed9c-41ad-b282-3032a15115e8 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:14.829360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18045b13-ed9c-41ad-b282-3032a15115e8 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:14.838834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40805c6-1695-4d2e-8221-c3d019c58997 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:14.839011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40805c6-1695-4d2e-8221-c3d019c58997 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:14.839256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40805c6-1695-4d2e-8221-c3d019c58997 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:14.839568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40805c6-1695-4d2e-8221-c3d019c58997 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:14.841740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e40805c6-1695-4d2e-8221-c3d019c58997 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:14.857636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40805c6-1695-4d2e-8221-c3d019c58997 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:14.861955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40805c6-1695-4d2e-8221-c3d019c58997 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:14.870657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1137/2284] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:15.545336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1540a-2450-4245-9242-5dc76e1d764a None None] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:15.546166 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1540a-2450-4245-9242-5dc76e1d764a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:15.546672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1540a-2450-4245-9242-5dc76e1d764a None None] PATH_INFO: `/` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:15.548807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1148/2285] 10.222.0.17 () {54 vars in 921 bytes} [Thu Aug 11 10:05:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 10:05:15.569897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf26056-3a9a-4968-aa97-d447cb556634 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:15.569897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf26056-3a9a-4968-aa97-d447cb556634 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:15.569897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf26056-3a9a-4968-aa97-d447cb556634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:15.583951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9cf26056-3a9a-4968-aa97-d447cb556634 None None] MFA Rules not processed for user `10ce125d46b545499de1fcf7072963e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:15.586017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9cf26056-3a9a-4968-aa97-d447cb556634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:15.597353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1138/2286] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Aug 11 10:05:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:15.615484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1540a-2450-4245-9242-5dc76e1d764a None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:15.618244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8da1540a-2450-4245-9242-5dc76e1d764a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:15.640388 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1540a-2450-4245-9242-5dc76e1d764a None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:15.650103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df61e72-2712-42df-9ee3-8e5f8899068a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:15.650772 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df61e72-2712-42df-9ee3-8e5f8899068a service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:15.650905 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df61e72-2712-42df-9ee3-8e5f8899068a service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:15.651622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df61e72-2712-42df-9ee3-8e5f8899068a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:15.661924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df61e72-2712-42df-9ee3-8e5f8899068a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:15.663533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df61e72-2712-42df-9ee3-8e5f8899068a service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:15.673913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1149/2287] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:15.703260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf26056-3a9a-4968-aa97-d447cb556634 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:15.705343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9cf26056-3a9a-4968-aa97-d447cb556634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:15.788301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf26056-3a9a-4968-aa97-d447cb556634 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:15.793259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e177300-d00f-4c12-9018-b69909b44779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:15.794012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177300-d00f-4c12-9018-b69909b44779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:15.794462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177300-d00f-4c12-9018-b69909b44779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:15.794739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e177300-d00f-4c12-9018-b69909b44779 admin admin] PATH_INFO: `/v3/users/acd4018e26564721a8c16db0e9d2e0aa` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:15.795949 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e177300-d00f-4c12-9018-b69909b44779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acd4018e26564721a8c16db0e9d2e0aa)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:15.797900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e177300-d00f-4c12-9018-b69909b44779 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:16.004477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9e177300-d00f-4c12-9018-b69909b44779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acd4018e26564721a8c16db0e9d2e0aa'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:16.005296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9e177300-d00f-4c12-9018-b69909b44779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acd4018e26564721a8c16db0e9d2e0aa'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:16.006097 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9e177300-d00f-4c12-9018-b69909b44779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acd4018e26564721a8c16db0e9d2e0aa'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:16.089657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9e177300-d00f-4c12-9018-b69909b44779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acd4018e26564721a8c16db0e9d2e0aa'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:16.109154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9e177300-d00f-4c12-9018-b69909b44779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acd4018e26564721a8c16db0e9d2e0aa'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:16.124880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1139/2288] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:05:15 2022] DELETE /identity/v3/users/acd4018e26564721a8c16db0e9d2e0aa => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:05:16.507984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df61e72-2712-42df-9ee3-8e5f8899068a service placement] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:16.535418 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df61e72-2712-42df-9ee3-8e5f8899068a service placement] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:16.629643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:16.632173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:16.632634 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:16.633253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] PATH_INFO: `/v3/projects/dd2a37c402a64171aae8a8c90b89ce33` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:16.635003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd2a37c402a64171aae8a8c90b89ce33)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:16.637228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:16.796759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd2a37c402a64171aae8a8c90b89ce33', 'request_id': 'req-d9793806-ed29-457a-86aa-b93c19b5fed1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:16.803026 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd2a37c402a64171aae8a8c90b89ce33', 'request_id': 'req-d9793806-ed29-457a-86aa-b93c19b5fed1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:16.810342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd2a37c402a64171aae8a8c90b89ce33', 'request_id': 'req-d9793806-ed29-457a-86aa-b93c19b5fed1'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:16.888626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] The token cache is being invalidate because project dd2a37c402a64171aae8a8c90b89ce33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:05:16.888828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:16.889539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:16.891053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1150/2289] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:05:16 2022] DELETE /identity/v3/projects/dd2a37c402a64171aae8a8c90b89ce33 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:17.006603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba977cf-5508-478d-bc44-2f77aae710bb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:17.007919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba977cf-5508-478d-bc44-2f77aae710bb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:17.008202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba977cf-5508-478d-bc44-2f77aae710bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:17.032143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0ba977cf-5508-478d-bc44-2f77aae710bb None None] MFA Rules not processed for user `6113630b377f40eda2f8994ae69633ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:17.033425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0ba977cf-5508-478d-bc44-2f77aae710bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:17.095093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1140/2290] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:17.403294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:17.404738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:17.462622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9793806-ed29-457a-86aa-b93c19b5fed1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:17.466179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6ab60-66b0-433d-ad47-f605949844ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:17.467042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6ab60-66b0-433d-ad47-f605949844ba service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:17.467321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6ab60-66b0-433d-ad47-f605949844ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:17.467593 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6ab60-66b0-433d-ad47-f605949844ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:17.469423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-23d6ab60-66b0-433d-ad47-f605949844ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:17.479191 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6ab60-66b0-433d-ad47-f605949844ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:17.480631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6ab60-66b0-433d-ad47-f605949844ba service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:17.488322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1151/2291] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:05:17.913542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba977cf-5508-478d-bc44-2f77aae710bb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:17.915132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0ba977cf-5508-478d-bc44-2f77aae710bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:17.921572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:17.988220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba977cf-5508-478d-bc44-2f77aae710bb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:17.993428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f6f0a-7ca9-4165-be16-c2858e9c8a47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:17.994061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f6f0a-7ca9-4165-be16-c2858e9c8a47 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:17.994344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f6f0a-7ca9-4165-be16-c2858e9c8a47 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:17.994609 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f6f0a-7ca9-4165-be16-c2858e9c8a47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:17.996343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-251f6f0a-7ca9-4165-be16-c2858e9c8a47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:18.008662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f6f0a-7ca9-4165-be16-c2858e9c8a47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:18.010314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f6f0a-7ca9-4165-be16-c2858e9c8a47 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:18.020153 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1141/2292] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:23.049309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6ab60-66b0-433d-ad47-f605949844ba service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:23.060279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6ab60-66b0-433d-ad47-f605949844ba service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:23.064088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc9021-52df-4497-b5bf-edc56884bc5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:23.064690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc9021-52df-4497-b5bf-edc56884bc5c service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:23.064971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc9021-52df-4497-b5bf-edc56884bc5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:23.065286 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc9021-52df-4497-b5bf-edc56884bc5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:23.067088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1cfc9021-52df-4497-b5bf-edc56884bc5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:23.073759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:23.077841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc9021-52df-4497-b5bf-edc56884bc5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:23.079295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc9021-52df-4497-b5bf-edc56884bc5c service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:23.087503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1152/2293] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:28.180229 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=30 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:29.691700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f6f0a-7ca9-4165-be16-c2858e9c8a47 service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:29.701575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f6f0a-7ca9-4165-be16-c2858e9c8a47 service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:29.705490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613883-ba48-4f25-a487-a4025d8904bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:29.706100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613883-ba48-4f25-a487-a4025d8904bb service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:29.706383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613883-ba48-4f25-a487-a4025d8904bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:29.706704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613883-ba48-4f25-a487-a4025d8904bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:29.708595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-40613883-ba48-4f25-a487-a4025d8904bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:29.783968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40613883-ba48-4f25-a487-a4025d8904bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:29.785446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40613883-ba48-4f25-a487-a4025d8904bb service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:29.800625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1142/2294] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:31.689918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc9021-52df-4497-b5bf-edc56884bc5c service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:31.692757 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1cfc9021-52df-4497-b5bf-edc56884bc5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:31.781201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc9021-52df-4497-b5bf-edc56884bc5c service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:31.789809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f1509-dce7-4587-bec7-e661ba8a0c79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:31.792312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f1509-dce7-4587-bec7-e661ba8a0c79 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:31.792585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f1509-dce7-4587-bec7-e661ba8a0c79 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:31.792866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f1509-dce7-4587-bec7-e661ba8a0c79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:31.795214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-906f1509-dce7-4587-bec7-e661ba8a0c79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:31.811050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f1509-dce7-4587-bec7-e661ba8a0c79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:31.812951 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f1509-dce7-4587-bec7-e661ba8a0c79 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:31.825590 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1153/2295] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:34.024142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613883-ba48-4f25-a487-a4025d8904bb service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:34.046091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613883-ba48-4f25-a487-a4025d8904bb service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:34.052582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7883313-be32-464b-afce-9f5dc1ec2550 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:34.057979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7883313-be32-464b-afce-9f5dc1ec2550 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:34.058480 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7883313-be32-464b-afce-9f5dc1ec2550 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:34.058984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7883313-be32-464b-afce-9f5dc1ec2550 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:34.061738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c7883313-be32-464b-afce-9f5dc1ec2550 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:34.076927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7883313-be32-464b-afce-9f5dc1ec2550 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:34.079204 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7883313-be32-464b-afce-9f5dc1ec2550 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:34.092639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1143/2296] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:34.221229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f1509-dce7-4587-bec7-e661ba8a0c79 service nova] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:34.223391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-906f1509-dce7-4587-bec7-e661ba8a0c79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:34.243675 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f1509-dce7-4587-bec7-e661ba8a0c79 service nova] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:34.248596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09bab1-bed5-40aa-ac8d-eeaa9745c2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:34.249272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09bab1-bed5-40aa-ac8d-eeaa9745c2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:34.249552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09bab1-bed5-40aa-ac8d-eeaa9745c2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:34.249829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09bab1-bed5-40aa-ac8d-eeaa9745c2d0 admin admin] PATH_INFO: `/v3/users/c62f05ea732142c1af1c09dd23e7b9a3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:34.251013 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09bab1-bed5-40aa-ac8d-eeaa9745c2d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c62f05ea732142c1af1c09dd23e7b9a3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:34.252943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09bab1-bed5-40aa-ac8d-eeaa9745c2d0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:34.533324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bb09bab1-bed5-40aa-ac8d-eeaa9745c2d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c62f05ea732142c1af1c09dd23e7b9a3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:34.541233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bb09bab1-bed5-40aa-ac8d-eeaa9745c2d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c62f05ea732142c1af1c09dd23e7b9a3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:34.603929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bb09bab1-bed5-40aa-ac8d-eeaa9745c2d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c62f05ea732142c1af1c09dd23e7b9a3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:34.604871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bb09bab1-bed5-40aa-ac8d-eeaa9745c2d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c62f05ea732142c1af1c09dd23e7b9a3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:34.619769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-bb09bab1-bed5-40aa-ac8d-eeaa9745c2d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c62f05ea732142c1af1c09dd23e7b9a3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:34.637291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1154/2297] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:05:34 2022] DELETE /identity/v3/users/c62f05ea732142c1af1c09dd23e7b9a3 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:34.913759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7883313-be32-464b-afce-9f5dc1ec2550 service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:34.928804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7883313-be32-464b-afce-9f5dc1ec2550 service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:35.028451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:35.029244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:35.029722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:35.030173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] PATH_INFO: `/v3/projects/75fb7bc39c364a5fb80532cbd3b2f654` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:35.031338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75fb7bc39c364a5fb80532cbd3b2f654)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:35.033259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:35.168668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75fb7bc39c364a5fb80532cbd3b2f654', 'request_id': 'req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:35.177837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75fb7bc39c364a5fb80532cbd3b2f654', 'request_id': 'req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:35.195604 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75fb7bc39c364a5fb80532cbd3b2f654', 'request_id': 'req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:35.265256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] The token cache is being invalidate because project 75fb7bc39c364a5fb80532cbd3b2f654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:05:35.265747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:35.266461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4b2fcb8f-b85f-47e8-9c06-4b499ffecc23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:35.268198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1144/2298] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:05:34 2022] DELETE /identity/v3/projects/75fb7bc39c364a5fb80532cbd3b2f654 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:35.278758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a6db7-a27c-4d21-9a25-271fa8b6a363 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:35.279279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a6db7-a27c-4d21-9a25-271fa8b6a363 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:35.279424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a6db7-a27c-4d21-9a25-271fa8b6a363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:35.300737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-694a6db7-a27c-4d21-9a25-271fa8b6a363 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:35.303748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-694a6db7-a27c-4d21-9a25-271fa8b6a363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:35.353815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:35.372787 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1155/2299] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:35.382504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4798d-087b-4f05-8689-f3d157162da8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:35.382941 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4798d-087b-4f05-8689-f3d157162da8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:35.383315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4798d-087b-4f05-8689-f3d157162da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:35.406167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-84e4798d-087b-4f05-8689-f3d157162da8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:35.407499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-84e4798d-087b-4f05-8689-f3d157162da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:35.416221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1145/2300] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:35.421041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a6db7-a27c-4d21-9a25-271fa8b6a363 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:35.432647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a6db7-a27c-4d21-9a25-271fa8b6a363 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:35.438652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895816a-20a9-40d4-89bc-890ecfc24a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:35.439285 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895816a-20a9-40d4-89bc-890ecfc24a1c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:35.439565 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895816a-20a9-40d4-89bc-890ecfc24a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:35.439834 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895816a-20a9-40d4-89bc-890ecfc24a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:35.440500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895816a-20a9-40d4-89bc-890ecfc24a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:05:35.440942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895816a-20a9-40d4-89bc-890ecfc24a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:35.442639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895816a-20a9-40d4-89bc-890ecfc24a1c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:35.456727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1156/2301] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:05:35.461569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4798d-087b-4f05-8689-f3d157162da8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:35.471332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:35.474512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4798d-087b-4f05-8689-f3d157162da8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:35.481820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57907de2-420e-45da-a7a5-c3ed8ae65564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:35.482789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57907de2-420e-45da-a7a5-c3ed8ae65564 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:35.483231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57907de2-420e-45da-a7a5-c3ed8ae65564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:35.483665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57907de2-420e-45da-a7a5-c3ed8ae65564 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:35.484529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57907de2-420e-45da-a7a5-c3ed8ae65564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:35.487279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57907de2-420e-45da-a7a5-c3ed8ae65564 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:35.577373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1146/2302] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:05:35 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:05:35.581365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895816a-20a9-40d4-89bc-890ecfc24a1c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:35.592801 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895816a-20a9-40d4-89bc-890ecfc24a1c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:35.669470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23814fdc-816d-4001-b48a-b6441187038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:35.670560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23814fdc-816d-4001-b48a-b6441187038c admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:35.670848 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23814fdc-816d-4001-b48a-b6441187038c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:35.671195 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23814fdc-816d-4001-b48a-b6441187038c admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:35.672016 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23814fdc-816d-4001-b48a-b6441187038c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:35.674716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23814fdc-816d-4001-b48a-b6441187038c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:35.785946 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1157/2303] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:05:35 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:05:35.790360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57907de2-420e-45da-a7a5-c3ed8ae65564 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:35.801470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57907de2-420e-45da-a7a5-c3ed8ae65564 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:35.809471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bbe0d-cfa6-42c6-8205-147b71c92ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:35.810353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bbe0d-cfa6-42c6-8205-147b71c92ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:35.810631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bbe0d-cfa6-42c6-8205-147b71c92ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:35.810894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bbe0d-cfa6-42c6-8205-147b71c92ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:35.811760 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bbe0d-cfa6-42c6-8205-147b71c92ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:35.813507 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bbe0d-cfa6-42c6-8205-147b71c92ae5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:35.826573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1147/2304] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:05:35.831062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23814fdc-816d-4001-b48a-b6441187038c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:35.842735 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23814fdc-816d-4001-b48a-b6441187038c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:35.853533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dcda3-67d3-488e-b147-65b20085a5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:35.854257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dcda3-67d3-488e-b147-65b20085a5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:35.854542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dcda3-67d3-488e-b147-65b20085a5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:35.854815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dcda3-67d3-488e-b147-65b20085a5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:35.856138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521dcda3-67d3-488e-b147-65b20085a5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:35.857755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521dcda3-67d3-488e-b147-65b20085a5e0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:35.871051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1158/2305] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:05:35.875833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bbe0d-cfa6-42c6-8205-147b71c92ae5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:35.888994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bbe0d-cfa6-42c6-8205-147b71c92ae5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:35.893824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1790a-80d0-4429-916d-edec74b36e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:35.894704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1790a-80d0-4429-916d-edec74b36e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:35.895088 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1790a-80d0-4429-916d-edec74b36e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:35.895461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1790a-80d0-4429-916d-edec74b36e2e admin admin] PATH_INFO: `/v3/projects/3a3d8d6884d84837aadcee25361207cd/users/d576231a16264588b9f183a6035bb3ae/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:35.913885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1790a-80d0-4429-916d-edec74b36e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a3d8d6884d84837aadcee25361207cd, user_id=d576231a16264588b9f183a6035bb3ae, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:35.916639 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1790a-80d0-4429-916d-edec74b36e2e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:35.997802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1148/2306] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:05:35 2022] PUT /identity/v3/projects/3a3d8d6884d84837aadcee25361207cd/users/d576231a16264588b9f183a6035bb3ae/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:36.009694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e47502-72fe-4639-a7fa-354592b1671b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:36.009989 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e47502-72fe-4639-a7fa-354592b1671b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:36.010315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e47502-72fe-4639-a7fa-354592b1671b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:36.039852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-53e47502-72fe-4639-a7fa-354592b1671b None None] MFA Rules not processed for user `d576231a16264588b9f183a6035bb3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:36.041350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-53e47502-72fe-4639-a7fa-354592b1671b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:36.117078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1159/2307] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:05:36.125552 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1790a-80d0-4429-916d-edec74b36e2e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:36.129830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-37f1790a-80d0-4429-916d-edec74b36e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:36.223888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1790a-80d0-4429-916d-edec74b36e2e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:36.230523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dfe24-ed69-45f1-a25c-23187de6ac85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:36.231211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dfe24-ed69-45f1-a25c-23187de6ac85 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:36.231540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dfe24-ed69-45f1-a25c-23187de6ac85 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:36.232317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dfe24-ed69-45f1-a25c-23187de6ac85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:36.240765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98dfe24-ed69-45f1-a25c-23187de6ac85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:36.242455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98dfe24-ed69-45f1-a25c-23187de6ac85 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:36.252083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1149/2308] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:36.480679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e47502-72fe-4639-a7fa-354592b1671b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:36.531737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:36.546898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e47502-72fe-4639-a7fa-354592b1671b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:36.551371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:36.556362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:36.556362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:36.556362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] PATH_INFO: `/v3/users/d576231a16264588b9f183a6035bb3ae` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:36.556362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d576231a16264588b9f183a6035bb3ae)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:36.556362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:36.674596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dfe24-ed69-45f1-a25c-23187de6ac85 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:36.691060 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dfe24-ed69-45f1-a25c-23187de6ac85 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:36.697983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c8cc4-258e-430d-8a73-4974786eaf53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:36.698671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c8cc4-258e-430d-8a73-4974786eaf53 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:36.699018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c8cc4-258e-430d-8a73-4974786eaf53 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:36.699297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c8cc4-258e-430d-8a73-4974786eaf53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:36.701065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1b2c8cc4-258e-430d-8a73-4974786eaf53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:36.718300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c8cc4-258e-430d-8a73-4974786eaf53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:36.720142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c8cc4-258e-430d-8a73-4974786eaf53 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:36.729352 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd576231a16264588b9f183a6035bb3ae'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:36.733269 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1150/2309] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:05:36.737279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd576231a16264588b9f183a6035bb3ae'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:36.813523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd576231a16264588b9f183a6035bb3ae'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:36.816836 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd576231a16264588b9f183a6035bb3ae'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:36.832089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd576231a16264588b9f183a6035bb3ae'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:36.839088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1160/2310] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:05:36 2022] DELETE /identity/v3/users/d576231a16264588b9f183a6035bb3ae => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:36.843759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6e64a-b8fa-41e5-9496-cffbceb82123 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:36.844043 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6e64a-b8fa-41e5-9496-cffbceb82123 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:36.844315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6e64a-b8fa-41e5-9496-cffbceb82123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:36.866042 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6ad6e64a-b8fa-41e5-9496-cffbceb82123 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:36.870530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6ad6e64a-b8fa-41e5-9496-cffbceb82123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:36.947797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1151/2311] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:36.957992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:36.962045 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:37.046088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0f0a2-e2a9-4b0f-a635-25022a24a643 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:37.050724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b67285-38c8-4d5a-bc44-b0da25afeb62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:37.051333 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b67285-38c8-4d5a-bc44-b0da25afeb62 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:37.051623 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b67285-38c8-4d5a-bc44-b0da25afeb62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:37.051975 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b67285-38c8-4d5a-bc44-b0da25afeb62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:37.067405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b67285-38c8-4d5a-bc44-b0da25afeb62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:37.070663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b67285-38c8-4d5a-bc44-b0da25afeb62 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:37.102291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1161/2312] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:37.675543 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6e64a-b8fa-41e5-9496-cffbceb82123 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:37.677753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6ad6e64a-b8fa-41e5-9496-cffbceb82123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:37.691437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6e64a-b8fa-41e5-9496-cffbceb82123 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:37.702169 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:37.702169 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:37.702169 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:37.702169 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] PATH_INFO: `/v3/projects/3a3d8d6884d84837aadcee25361207cd` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:37.702799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a3d8d6884d84837aadcee25361207cd)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:37.703147 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:37.721966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:37.822327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a3d8d6884d84837aadcee25361207cd', 'request_id': 'req-3c68d507-fb00-44a8-904b-686d0a95e7bd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:37.828920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a3d8d6884d84837aadcee25361207cd', 'request_id': 'req-3c68d507-fb00-44a8-904b-686d0a95e7bd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:37.837184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a3d8d6884d84837aadcee25361207cd', 'request_id': 'req-3c68d507-fb00-44a8-904b-686d0a95e7bd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:37.926008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] The token cache is being invalidate because project 3a3d8d6884d84837aadcee25361207cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:05:37.926328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:37.927012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3c68d507-fb00-44a8-904b-686d0a95e7bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:37.928570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1152/2313] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:05:37 2022] DELETE /identity/v3/projects/3a3d8d6884d84837aadcee25361207cd => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:37.933591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72ff8-5e10-43a7-b85a-304c9c1385f5 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:37.934161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72ff8-5e10-43a7-b85a-304c9c1385f5 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:37.934161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72ff8-5e10-43a7-b85a-304c9c1385f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:37.955219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f7f72ff8-5e10-43a7-b85a-304c9c1385f5 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:37.956458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f7f72ff8-5e10-43a7-b85a-304c9c1385f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:38.023364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1162/2314] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:05:38.031240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832f16-3784-4fc4-9d26-272a08131023 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:38.031468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832f16-3784-4fc4-9d26-272a08131023 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:38.031746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a832f16-3784-4fc4-9d26-272a08131023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:38.052833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2a832f16-3784-4fc4-9d26-272a08131023 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:38.054455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2a832f16-3784-4fc4-9d26-272a08131023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:38.062493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1153/2315] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:38.066913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f72ff8-5e10-43a7-b85a-304c9c1385f5 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:38.077808 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f72ff8-5e10-43a7-b85a-304c9c1385f5 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:38.082453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb839510-e955-4970-8d8e-0a892a5ec908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:38.083056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb839510-e955-4970-8d8e-0a892a5ec908 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:38.083370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb839510-e955-4970-8d8e-0a892a5ec908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:38.084209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb839510-e955-4970-8d8e-0a892a5ec908 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:38.084866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb839510-e955-4970-8d8e-0a892a5ec908 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:05:38.085211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb839510-e955-4970-8d8e-0a892a5ec908 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:38.087001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb839510-e955-4970-8d8e-0a892a5ec908 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:38.089832 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:38.099436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1163/2316] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:05:38.104177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a832f16-3784-4fc4-9d26-272a08131023 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:38.113778 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a832f16-3784-4fc4-9d26-272a08131023 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:38.118879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac15b9-0d2d-492e-9c95-a8ebe0a2713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:38.121250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac15b9-0d2d-492e-9c95-a8ebe0a2713c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:38.121250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac15b9-0d2d-492e-9c95-a8ebe0a2713c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:38.121250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac15b9-0d2d-492e-9c95-a8ebe0a2713c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:38.121250 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac15b9-0d2d-492e-9c95-a8ebe0a2713c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:38.122759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac15b9-0d2d-492e-9c95-a8ebe0a2713c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:38.262095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1154/2317] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:05:38 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:05:38.266372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb839510-e955-4970-8d8e-0a892a5ec908 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:38.278150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb839510-e955-4970-8d8e-0a892a5ec908 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:38.336635 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19b2fa-1620-443b-852f-b69396b451fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:38.337271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19b2fa-1620-443b-852f-b69396b451fc admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:38.337552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19b2fa-1620-443b-852f-b69396b451fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:38.337823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19b2fa-1620-443b-852f-b69396b451fc admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:38.352422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19b2fa-1620-443b-852f-b69396b451fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:38.352422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19b2fa-1620-443b-852f-b69396b451fc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:38.425630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1164/2318] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:05:38 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:05:38.430477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac15b9-0d2d-492e-9c95-a8ebe0a2713c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:38.440536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac15b9-0d2d-492e-9c95-a8ebe0a2713c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:38.447535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d43c0a-de4b-4557-92aa-a96a4678d674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:38.448146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43c0a-de4b-4557-92aa-a96a4678d674 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:38.448422 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43c0a-de4b-4557-92aa-a96a4678d674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:38.448696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43c0a-de4b-4557-92aa-a96a4678d674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:38.449375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d43c0a-de4b-4557-92aa-a96a4678d674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:38.450980 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d43c0a-de4b-4557-92aa-a96a4678d674 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:38.464240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1155/2319] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:05:38.465522 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19b2fa-1620-443b-852f-b69396b451fc admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:38.476130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19b2fa-1620-443b-852f-b69396b451fc admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:38.480993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21020613-f9f6-48c4-a65f-e2ffc4b12671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:38.484329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21020613-f9f6-48c4-a65f-e2ffc4b12671 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:38.484329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21020613-f9f6-48c4-a65f-e2ffc4b12671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:38.484329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21020613-f9f6-48c4-a65f-e2ffc4b12671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:38.484329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21020613-f9f6-48c4-a65f-e2ffc4b12671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:38.484683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21020613-f9f6-48c4-a65f-e2ffc4b12671 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:38.497495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1165/2320] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:05:38.501720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d43c0a-de4b-4557-92aa-a96a4678d674 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:38.512685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d43c0a-de4b-4557-92aa-a96a4678d674 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:38.518463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff237a-4813-4866-8689-82046b0611b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:38.519156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff237a-4813-4866-8689-82046b0611b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:38.519539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff237a-4813-4866-8689-82046b0611b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:38.519813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff237a-4813-4866-8689-82046b0611b1 admin admin] PATH_INFO: `/v3/projects/19cfcbe33ad74325a140912a37eed75c/users/9dc367642ed942bbb66ac79add731534/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:38.536343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff237a-4813-4866-8689-82046b0611b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19cfcbe33ad74325a140912a37eed75c, user_id=9dc367642ed942bbb66ac79add731534, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:38.539054 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff237a-4813-4866-8689-82046b0611b1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:38.625938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1156/2321] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:05:38 2022] PUT /identity/v3/projects/19cfcbe33ad74325a140912a37eed75c/users/9dc367642ed942bbb66ac79add731534/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:38.636629 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fc569-6eb1-453b-abdf-a73e6bee5ab2 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:38.636915 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fc569-6eb1-453b-abdf-a73e6bee5ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:38.637218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fc569-6eb1-453b-abdf-a73e6bee5ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:38.666644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c47fc569-6eb1-453b-abdf-a73e6bee5ab2 None None] MFA Rules not processed for user `9dc367642ed942bbb66ac79add731534`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:38.667943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c47fc569-6eb1-453b-abdf-a73e6bee5ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:38.732788 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1166/2322] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:38.737468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff237a-4813-4866-8689-82046b0611b1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:38.803650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff237a-4813-4866-8689-82046b0611b1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:38.808586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2adffb-6db1-4efa-95df-6f8dc10cfc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:38.809361 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2adffb-6db1-4efa-95df-6f8dc10cfc7d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:38.809640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2adffb-6db1-4efa-95df-6f8dc10cfc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:38.809912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2adffb-6db1-4efa-95df-6f8dc10cfc7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:38.810535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2adffb-6db1-4efa-95df-6f8dc10cfc7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:38.812418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2adffb-6db1-4efa-95df-6f8dc10cfc7d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:38.850320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d8232-bf87-4e32-9611-a8daab83aa21 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:38.850605 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d8232-bf87-4e32-9611-a8daab83aa21 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:38.850878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d8232-bf87-4e32-9611-a8daab83aa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:38.869398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-234d8232-bf87-4e32-9611-a8daab83aa21 None None] MFA Rules not processed for user `28b31650305943b498f02c882baa0144`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:38.870663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-234d8232-bf87-4e32-9611-a8daab83aa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:38.906872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1157/2323] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:05:38 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:05:38.910728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2adffb-6db1-4efa-95df-6f8dc10cfc7d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:38.931271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1167/2324] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:38.938219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:38.977228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2adffb-6db1-4efa-95df-6f8dc10cfc7d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:38.983031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a8aad-9572-46c5-9c43-7556c4ae5c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:38.983031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a8aad-9572-46c5-9c43-7556c4ae5c02 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:38.983031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a8aad-9572-46c5-9c43-7556c4ae5c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:38.983031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a8aad-9572-46c5-9c43-7556c4ae5c02 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:38.983728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a8aad-9572-46c5-9c43-7556c4ae5c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:38.985668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a8aad-9572-46c5-9c43-7556c4ae5c02 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:39.054316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1158/2325] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:05:38 2022] POST /identity/v3/users => generated 297 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:05:39.058404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d8232-bf87-4e32-9611-a8daab83aa21 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:39.069283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d8232-bf87-4e32-9611-a8daab83aa21 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:39.074768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088aa0a-787d-40f2-9e85-26cc7259f02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:39.075451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088aa0a-787d-40f2-9e85-26cc7259f02a admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:39.075737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088aa0a-787d-40f2-9e85-26cc7259f02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:39.076030 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088aa0a-787d-40f2-9e85-26cc7259f02a admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:39.076662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088aa0a-787d-40f2-9e85-26cc7259f02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:39.078301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088aa0a-787d-40f2-9e85-26cc7259f02a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:39.080553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:39.088983 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1168/2326] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:05:39.093624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a8aad-9572-46c5-9c43-7556c4ae5c02 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:39.105242 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a8aad-9572-46c5-9c43-7556c4ae5c02 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:39.109977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f589bac-77bb-4c7f-9ced-9b65a2c0448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:39.110708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f589bac-77bb-4c7f-9ced-9b65a2c0448f admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:39.111058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f589bac-77bb-4c7f-9ced-9b65a2c0448f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:39.111419 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f589bac-77bb-4c7f-9ced-9b65a2c0448f admin admin] PATH_INFO: `/v3/projects/7731a269e05742e5b56a23b71845ba6f/users/b154996141764b10850948bc8ead0951/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:39.127136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f589bac-77bb-4c7f-9ced-9b65a2c0448f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7731a269e05742e5b56a23b71845ba6f, user_id=b154996141764b10850948bc8ead0951, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:39.129866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f589bac-77bb-4c7f-9ced-9b65a2c0448f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:39.197090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1159/2327] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:05:39 2022] PUT /identity/v3/projects/7731a269e05742e5b56a23b71845ba6f/users/b154996141764b10850948bc8ead0951/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:39.201053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088aa0a-787d-40f2-9e85-26cc7259f02a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:39.212125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088aa0a-787d-40f2-9e85-26cc7259f02a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:39.283410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca317230-8b91-4479-b4c9-0164c088c104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:39.284086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317230-8b91-4479-b4c9-0164c088c104 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:39.284377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317230-8b91-4479-b4c9-0164c088c104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:39.284648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317230-8b91-4479-b4c9-0164c088c104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:39.285327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317230-8b91-4479-b4c9-0164c088c104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:39.287911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317230-8b91-4479-b4c9-0164c088c104 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:39.298085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1169/2328] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:05:39.302326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f589bac-77bb-4c7f-9ced-9b65a2c0448f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:39.312844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f589bac-77bb-4c7f-9ced-9b65a2c0448f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:39.317783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242a09e-9bb6-4f15-a923-5dfa03d7e1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:39.318667 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242a09e-9bb6-4f15-a923-5dfa03d7e1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:39.319012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242a09e-9bb6-4f15-a923-5dfa03d7e1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:39.319355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242a09e-9bb6-4f15-a923-5dfa03d7e1bc admin admin] PATH_INFO: `/v3/projects/7731a269e05742e5b56a23b71845ba6f/users/b154996141764b10850948bc8ead0951/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:39.321369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242a09e-9bb6-4f15-a923-5dfa03d7e1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7731a269e05742e5b56a23b71845ba6f, user_id=b154996141764b10850948bc8ead0951, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:39.323980 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242a09e-9bb6-4f15-a923-5dfa03d7e1bc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:39.335298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1160/2329] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:05:39 2022] PUT /identity/v3/projects/7731a269e05742e5b56a23b71845ba6f/users/b154996141764b10850948bc8ead0951/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:39.346738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d44417-0001-4869-93b2-c94fba326493 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:39.347062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d44417-0001-4869-93b2-c94fba326493 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:39.347344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d44417-0001-4869-93b2-c94fba326493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:39.376353 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-85d44417-0001-4869-93b2-c94fba326493 None None] MFA Rules not processed for user `b154996141764b10850948bc8ead0951`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:39.377631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-85d44417-0001-4869-93b2-c94fba326493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:39.448074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1170/2330] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:39.461729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242a09e-9bb6-4f15-a923-5dfa03d7e1bc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:39.463556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3242a09e-9bb6-4f15-a923-5dfa03d7e1bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:39.523700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242a09e-9bb6-4f15-a923-5dfa03d7e1bc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:39.527698 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b23e3-117e-4809-a263-646d306fa572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:39.528349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b23e3-117e-4809-a263-646d306fa572 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:39.528632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b23e3-117e-4809-a263-646d306fa572 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:39.528905 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b23e3-117e-4809-a263-646d306fa572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:39.538801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b23e3-117e-4809-a263-646d306fa572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:39.540237 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b23e3-117e-4809-a263-646d306fa572 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:39.550009 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1161/2331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:40.444614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d44417-0001-4869-93b2-c94fba326493 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:40.456770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d44417-0001-4869-93b2-c94fba326493 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:40.462368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec371d07-d85d-477a-a510-f7f0055a53ad tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b154996141764b10850948bc8ead0951', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7731a269e05742e5b56a23b71845ba6f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:40.463054 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec371d07-d85d-477a-a510-f7f0055a53ad tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:40.463387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec371d07-d85d-477a-a510-f7f0055a53ad tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:40.463669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec371d07-d85d-477a-a510-f7f0055a53ad tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:40.464331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec371d07-d85d-477a-a510-f7f0055a53ad tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:05:40.464637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec371d07-d85d-477a-a510-f7f0055a53ad tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:40.466426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec371d07-d85d-477a-a510-f7f0055a53ad tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:40.480469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1171/2332] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:05:40.484856 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b23e3-117e-4809-a263-646d306fa572 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:40.496442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b23e3-117e-4809-a263-646d306fa572 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:40.503497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22a104-b25c-4ca8-89f6-01d6768781a0 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b154996141764b10850948bc8ead0951', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7731a269e05742e5b56a23b71845ba6f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:40.504445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22a104-b25c-4ca8-89f6-01d6768781a0 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:40.504904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22a104-b25c-4ca8-89f6-01d6768781a0 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:40.505409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22a104-b25c-4ca8-89f6-01d6768781a0 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:40.506120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22a104-b25c-4ca8-89f6-01d6768781a0 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:40.508015 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22a104-b25c-4ca8-89f6-01d6768781a0 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:40.614094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1162/2333] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:05:40 2022] POST /identity/v3/projects => generated 355 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:05:40.615994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec371d07-d85d-477a-a510-f7f0055a53ad tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:40.627844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec371d07-d85d-477a-a510-f7f0055a53ad tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:40.692778 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29406e-8c00-4d2a-86ff-d31d12113bb6 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b154996141764b10850948bc8ead0951', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7731a269e05742e5b56a23b71845ba6f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:40.693646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29406e-8c00-4d2a-86ff-d31d12113bb6 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:40.693934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29406e-8c00-4d2a-86ff-d31d12113bb6 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:40.694216 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c29406e-8c00-4d2a-86ff-d31d12113bb6 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:40.694874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29406e-8c00-4d2a-86ff-d31d12113bb6 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:05:40.695183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29406e-8c00-4d2a-86ff-d31d12113bb6 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:40.696777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c29406e-8c00-4d2a-86ff-d31d12113bb6 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:40.705714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:40.710028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1172/2334] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:05:40.749349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22a104-b25c-4ca8-89f6-01d6768781a0 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:40.765303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22a104-b25c-4ca8-89f6-01d6768781a0 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:40.771956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b154996141764b10850948bc8ead0951', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7731a269e05742e5b56a23b71845ba6f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:40.772718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:40.773452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:40.773452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] PATH_INFO: `/v3/projects/d809d8fe9ee54cad8b6f9ae50d39ef7d` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:40.774531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d809d8fe9ee54cad8b6f9ae50d39ef7d)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:40.776455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:40.918968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd809d8fe9ee54cad8b6f9ae50d39ef7d', 'request_id': 'req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:40.920915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:40.925095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd809d8fe9ee54cad8b6f9ae50d39ef7d', 'request_id': 'req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:40.932110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd809d8fe9ee54cad8b6f9ae50d39ef7d', 'request_id': 'req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:41.008754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] The token cache is being invalidate because project d809d8fe9ee54cad8b6f9ae50d39ef7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:05:41.008962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:41.009630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2322e9ce-6d7e-4b12-8f7f-d163b6462a7f tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:41.011168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1163/2335] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:05:40 2022] DELETE /identity/v3/projects/d809d8fe9ee54cad8b6f9ae50d39ef7d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:41.721276 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29406e-8c00-4d2a-86ff-d31d12113bb6 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:41.723065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0c29406e-8c00-4d2a-86ff-d31d12113bb6 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:41.789237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c29406e-8c00-4d2a-86ff-d31d12113bb6 tempest-VolumeQuotaClassesTest-1344767613 tempest-VolumeQuotaClassesTest-1344767613-project-admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:41.849597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:41.850485 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:41.850767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:41.851043 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] PATH_INFO: `/v3/users/9dc367642ed942bbb66ac79add731534` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:41.852186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc367642ed942bbb66ac79add731534)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:41.854147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:42.016370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc367642ed942bbb66ac79add731534'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:42.023323 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc367642ed942bbb66ac79add731534'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:42.105155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc367642ed942bbb66ac79add731534'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:42.106345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc367642ed942bbb66ac79add731534'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:42.120199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc367642ed942bbb66ac79add731534'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:42.127378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1173/2336] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:05:41 2022] DELETE /identity/v3/users/9dc367642ed942bbb66ac79add731534 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:42.132357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0aead-36a2-4c33-8e02-2c502244c73b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:42.132852 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0aead-36a2-4c33-8e02-2c502244c73b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:42.133154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0aead-36a2-4c33-8e02-2c502244c73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:42.154174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c4b0aead-36a2-4c33-8e02-2c502244c73b None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:42.155410 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c4b0aead-36a2-4c33-8e02-2c502244c73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:42.227000 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1164/2337] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:42.237519 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:42.240953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:42.279591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:42.337820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60e3ea-e99f-44ac-9b4e-a33382dfc43d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:42.341529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871e74d-e1f6-421e-9eeb-88a7482c491b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:42.342396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871e74d-e1f6-421e-9eeb-88a7482c491b service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:42.342676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871e74d-e1f6-421e-9eeb-88a7482c491b service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:42.343018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871e74d-e1f6-421e-9eeb-88a7482c491b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:42.353195 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871e74d-e1f6-421e-9eeb-88a7482c491b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:42.354699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871e74d-e1f6-421e-9eeb-88a7482c491b service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:42.364067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1174/2338] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:42.815281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0aead-36a2-4c33-8e02-2c502244c73b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:42.816798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c4b0aead-36a2-4c33-8e02-2c502244c73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:42.830659 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0aead-36a2-4c33-8e02-2c502244c73b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:42.835689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:42.836384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:42.836663 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:42.837141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] PATH_INFO: `/v3/projects/19cfcbe33ad74325a140912a37eed75c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:42.838460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19cfcbe33ad74325a140912a37eed75c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:42.840368 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:42.979676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19cfcbe33ad74325a140912a37eed75c', 'request_id': 'req-d94ee8db-544c-41ca-94c5-90b8eddecc97'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:42.997984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19cfcbe33ad74325a140912a37eed75c', 'request_id': 'req-d94ee8db-544c-41ca-94c5-90b8eddecc97'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:43.007000 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19cfcbe33ad74325a140912a37eed75c', 'request_id': 'req-d94ee8db-544c-41ca-94c5-90b8eddecc97'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:43.116867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] The token cache is being invalidate because project 19cfcbe33ad74325a140912a37eed75c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:05:43.117315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:43.117968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:43.119481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1165/2339] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:05:42 2022] DELETE /identity/v3/projects/19cfcbe33ad74325a140912a37eed75c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:43.123751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871e74d-e1f6-421e-9eeb-88a7482c491b service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:43.134764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871e74d-e1f6-421e-9eeb-88a7482c491b service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:43.199434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917df834-5e2a-48e5-8bb5-b06ae3c07385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:43.201306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917df834-5e2a-48e5-8bb5-b06ae3c07385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:43.202133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917df834-5e2a-48e5-8bb5-b06ae3c07385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:43.202133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917df834-5e2a-48e5-8bb5-b06ae3c07385 admin admin] PATH_INFO: `/v3/users/b154996141764b10850948bc8ead0951` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:43.203266 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917df834-5e2a-48e5-8bb5-b06ae3c07385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b154996141764b10850948bc8ead0951)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:43.205283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917df834-5e2a-48e5-8bb5-b06ae3c07385 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:43.425962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-917df834-5e2a-48e5-8bb5-b06ae3c07385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b154996141764b10850948bc8ead0951'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:43.434470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-917df834-5e2a-48e5-8bb5-b06ae3c07385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b154996141764b10850948bc8ead0951'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:43.499650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-917df834-5e2a-48e5-8bb5-b06ae3c07385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b154996141764b10850948bc8ead0951'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:43.500284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-917df834-5e2a-48e5-8bb5-b06ae3c07385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b154996141764b10850948bc8ead0951'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:43.514560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-917df834-5e2a-48e5-8bb5-b06ae3c07385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b154996141764b10850948bc8ead0951'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:43.522416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1175/2340] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:05:43 2022] DELETE /identity/v3/users/b154996141764b10850948bc8ead0951 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:44.235609 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:44.237089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:44.250837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee8db-544c-41ca-94c5-90b8eddecc97 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:44.348508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:44.348508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:44.348508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:44.348508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] PATH_INFO: `/v3/projects/7731a269e05742e5b56a23b71845ba6f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:44.349080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7731a269e05742e5b56a23b71845ba6f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:44.349080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:44.356442 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:44.482592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7731a269e05742e5b56a23b71845ba6f', 'request_id': 'req-555fe5a9-2107-4b54-b758-59fdec077245'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:44.490026 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7731a269e05742e5b56a23b71845ba6f', 'request_id': 'req-555fe5a9-2107-4b54-b758-59fdec077245'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:44.500212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7731a269e05742e5b56a23b71845ba6f', 'request_id': 'req-555fe5a9-2107-4b54-b758-59fdec077245'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:44.586018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] The token cache is being invalidate because project 7731a269e05742e5b56a23b71845ba6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:05:44.586355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:44.587228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-555fe5a9-2107-4b54-b758-59fdec077245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:44.589134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1166/2341] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:05:44 2022] DELETE /identity/v3/projects/7731a269e05742e5b56a23b71845ba6f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:44.594510 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae20fc-0798-4322-ba76-55c5380789a1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:44.595005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae20fc-0798-4322-ba76-55c5380789a1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:44.595086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae20fc-0798-4322-ba76-55c5380789a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:44.618395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-d7ae20fc-0798-4322-ba76-55c5380789a1 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:44.619721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d7ae20fc-0798-4322-ba76-55c5380789a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:44.674218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:44.692497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1176/2342] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:44.700500 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cbd3c-b853-4df5-badc-050e0c2a51f6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:44.700856 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cbd3c-b853-4df5-badc-050e0c2a51f6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:44.701236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cbd3c-b853-4df5-badc-050e0c2a51f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:44.722376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4c0cbd3c-b853-4df5-badc-050e0c2a51f6 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:44.723736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4c0cbd3c-b853-4df5-badc-050e0c2a51f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:44.732558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1167/2343] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:44.736644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae20fc-0798-4322-ba76-55c5380789a1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:44.748789 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae20fc-0798-4322-ba76-55c5380789a1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:44.754149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478377f9-d72d-4f6c-8fd3-4a95802abd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:44.754765 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478377f9-d72d-4f6c-8fd3-4a95802abd89 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:44.755688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478377f9-d72d-4f6c-8fd3-4a95802abd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:44.755953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478377f9-d72d-4f6c-8fd3-4a95802abd89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:44.756879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478377f9-d72d-4f6c-8fd3-4a95802abd89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:05:44.756879 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478377f9-d72d-4f6c-8fd3-4a95802abd89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:44.758547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478377f9-d72d-4f6c-8fd3-4a95802abd89 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:44.774681 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1177/2344] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:05:44.779102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cbd3c-b853-4df5-badc-050e0c2a51f6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:44.789944 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cbd3c-b853-4df5-badc-050e0c2a51f6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:44.795575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee983514-7c53-49be-8a19-8fe041f33a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:44.796729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee983514-7c53-49be-8a19-8fe041f33a85 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:44.796729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee983514-7c53-49be-8a19-8fe041f33a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:44.798928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee983514-7c53-49be-8a19-8fe041f33a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:44.798928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee983514-7c53-49be-8a19-8fe041f33a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:44.799646 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee983514-7c53-49be-8a19-8fe041f33a85 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:44.906658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1168/2345] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:05:44 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:05:44.910976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478377f9-d72d-4f6c-8fd3-4a95802abd89 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:44.922200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478377f9-d72d-4f6c-8fd3-4a95802abd89 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:44.986936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922fb0c-1a18-46be-b55f-15f5e93bb42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:44.987560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922fb0c-1a18-46be-b55f-15f5e93bb42e admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:44.988271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922fb0c-1a18-46be-b55f-15f5e93bb42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:44.988271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922fb0c-1a18-46be-b55f-15f5e93bb42e admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:44.989104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922fb0c-1a18-46be-b55f-15f5e93bb42e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:44.991871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922fb0c-1a18-46be-b55f-15f5e93bb42e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:45.100429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1178/2346] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:05:44 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:05:45.104723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee983514-7c53-49be-8a19-8fe041f33a85 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:45.116260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee983514-7c53-49be-8a19-8fe041f33a85 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:45.121777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90b3e0-ff70-46d4-8213-57fd7db6cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:45.125736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90b3e0-ff70-46d4-8213-57fd7db6cd8c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:45.125736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90b3e0-ff70-46d4-8213-57fd7db6cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:45.125736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90b3e0-ff70-46d4-8213-57fd7db6cd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:45.125736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90b3e0-ff70-46d4-8213-57fd7db6cd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:45.125736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90b3e0-ff70-46d4-8213-57fd7db6cd8c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:45.135647 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1169/2347] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:05:45.139842 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922fb0c-1a18-46be-b55f-15f5e93bb42e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:45.150871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922fb0c-1a18-46be-b55f-15f5e93bb42e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:45.155694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea97df8-8011-4dd0-b95d-88eb73e23df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:45.156325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea97df8-8011-4dd0-b95d-88eb73e23df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:45.156608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea97df8-8011-4dd0-b95d-88eb73e23df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:45.156878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea97df8-8011-4dd0-b95d-88eb73e23df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:45.157587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea97df8-8011-4dd0-b95d-88eb73e23df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:45.159195 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea97df8-8011-4dd0-b95d-88eb73e23df3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:45.174020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1179/2348] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:05:45.175370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90b3e0-ff70-46d4-8213-57fd7db6cd8c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:45.186309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90b3e0-ff70-46d4-8213-57fd7db6cd8c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:45.192012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295d228-c591-4b35-abc7-d0da088361f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:45.194182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295d228-c591-4b35-abc7-d0da088361f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:45.196411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295d228-c591-4b35-abc7-d0da088361f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:45.196855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295d228-c591-4b35-abc7-d0da088361f1 admin admin] PATH_INFO: `/v3/projects/1a1fc6a4bd7a410597921cc0a74b0d18/users/b966c60cd2f942d78b644ff6598f38d8/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:45.217524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295d228-c591-4b35-abc7-d0da088361f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a1fc6a4bd7a410597921cc0a74b0d18, user_id=b966c60cd2f942d78b644ff6598f38d8, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:45.217524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295d228-c591-4b35-abc7-d0da088361f1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:45.329182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1170/2349] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:05:45 2022] PUT /identity/v3/projects/1a1fc6a4bd7a410597921cc0a74b0d18/users/b966c60cd2f942d78b644ff6598f38d8/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:45.340047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ac383-6fbb-41bf-b431-3e58cc8641c1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:45.340332 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ac383-6fbb-41bf-b431-3e58cc8641c1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:45.340598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ac383-6fbb-41bf-b431-3e58cc8641c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:45.373256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-af8ac383-6fbb-41bf-b431-3e58cc8641c1 None None] MFA Rules not processed for user `b966c60cd2f942d78b644ff6598f38d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:45.374395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-af8ac383-6fbb-41bf-b431-3e58cc8641c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:45.456213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1180/2350] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:45.460825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295d228-c591-4b35-abc7-d0da088361f1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:45.527551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295d228-c591-4b35-abc7-d0da088361f1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:45.535434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6483fa2-4606-420e-9a57-f9a86ca15c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:45.535434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6483fa2-4606-420e-9a57-f9a86ca15c4f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:45.535434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6483fa2-4606-420e-9a57-f9a86ca15c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:45.535434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6483fa2-4606-420e-9a57-f9a86ca15c4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:45.535786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6483fa2-4606-420e-9a57-f9a86ca15c4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:45.537044 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6483fa2-4606-420e-9a57-f9a86ca15c4f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:45.616497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1171/2351] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:05:45 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:05:45.620540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ac383-6fbb-41bf-b431-3e58cc8641c1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:45.631687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ac383-6fbb-41bf-b431-3e58cc8641c1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:45.682564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:45.697763 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe46c16-b227-4d76-9b59-544030c6db2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:45.698676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46c16-b227-4d76-9b59-544030c6db2f admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:45.698954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46c16-b227-4d76-9b59-544030c6db2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:45.699223 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe46c16-b227-4d76-9b59-544030c6db2f admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:45.699937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe46c16-b227-4d76-9b59-544030c6db2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:45.701958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe46c16-b227-4d76-9b59-544030c6db2f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:45.774808 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1181/2352] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:05:45 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:05:45.778728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6483fa2-4606-420e-9a57-f9a86ca15c4f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:45.789736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6483fa2-4606-420e-9a57-f9a86ca15c4f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:45.794435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3529948-7f18-4692-b8e7-6c35cfdedbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:45.795040 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529948-7f18-4692-b8e7-6c35cfdedbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:45.795318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529948-7f18-4692-b8e7-6c35cfdedbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:45.795592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529948-7f18-4692-b8e7-6c35cfdedbe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:45.796233 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3529948-7f18-4692-b8e7-6c35cfdedbe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:45.797858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3529948-7f18-4692-b8e7-6c35cfdedbe6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:45.809554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1172/2353] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:05:45.813796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe46c16-b227-4d76-9b59-544030c6db2f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:45.823947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe46c16-b227-4d76-9b59-544030c6db2f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:45.828715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d57cb9-9b36-404d-bda5-51272a597580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:45.829379 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d57cb9-9b36-404d-bda5-51272a597580 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:45.835433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d57cb9-9b36-404d-bda5-51272a597580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:45.835433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d57cb9-9b36-404d-bda5-51272a597580 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:45.835433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d57cb9-9b36-404d-bda5-51272a597580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:45.835433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d57cb9-9b36-404d-bda5-51272a597580 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:45.842666 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1182/2354] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:05:45.846611 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3529948-7f18-4692-b8e7-6c35cfdedbe6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:45.853471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:45.857107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3529948-7f18-4692-b8e7-6c35cfdedbe6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:45.861776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b6fbb-c761-4dcd-98dc-bbded6900c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:45.862420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b6fbb-c761-4dcd-98dc-bbded6900c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:45.863248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b6fbb-c761-4dcd-98dc-bbded6900c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:45.863248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b6fbb-c761-4dcd-98dc-bbded6900c4a admin admin] PATH_INFO: `/v3/projects/a34d40b7cc064e59a8ae2171d0b4c812/users/b59e55550a1644e38db5fe2fe1945b9f/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:45.878422 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b6fbb-c761-4dcd-98dc-bbded6900c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a34d40b7cc064e59a8ae2171d0b4c812, user_id=b59e55550a1644e38db5fe2fe1945b9f, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:45.881133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b6fbb-c761-4dcd-98dc-bbded6900c4a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:45.945566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1173/2355] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:05:45 2022] PUT /identity/v3/projects/a34d40b7cc064e59a8ae2171d0b4c812/users/b59e55550a1644e38db5fe2fe1945b9f/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:45.956857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455f177-7b86-4f2f-86d3-df0311d878bb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:45.957190 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455f177-7b86-4f2f-86d3-df0311d878bb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:45.957494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455f177-7b86-4f2f-86d3-df0311d878bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:45.986536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0455f177-7b86-4f2f-86d3-df0311d878bb None None] MFA Rules not processed for user `b59e55550a1644e38db5fe2fe1945b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:45.987777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0455f177-7b86-4f2f-86d3-df0311d878bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:46.055039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1183/2356] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:46.059622 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b6fbb-c761-4dcd-98dc-bbded6900c4a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:46.122566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b6fbb-c761-4dcd-98dc-bbded6900c4a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:46.127350 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875fa2e-5e86-43cb-8521-5cb32c78b956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:46.127959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875fa2e-5e86-43cb-8521-5cb32c78b956 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:46.128241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875fa2e-5e86-43cb-8521-5cb32c78b956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:46.128515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875fa2e-5e86-43cb-8521-5cb32c78b956 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:46.129175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875fa2e-5e86-43cb-8521-5cb32c78b956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:46.131115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875fa2e-5e86-43cb-8521-5cb32c78b956 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:46.198228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1174/2357] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:05:46 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:05:46.202378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455f177-7b86-4f2f-86d3-df0311d878bb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:46.213793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455f177-7b86-4f2f-86d3-df0311d878bb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:46.271372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e7a75-f181-41ef-ba85-7d2c8cfecef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:46.271976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e7a75-f181-41ef-ba85-7d2c8cfecef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:46.272257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e7a75-f181-41ef-ba85-7d2c8cfecef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:46.272526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e7a75-f181-41ef-ba85-7d2c8cfecef7 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:46.273183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e7a75-f181-41ef-ba85-7d2c8cfecef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:46.275095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e7a75-f181-41ef-ba85-7d2c8cfecef7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:46.362601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1184/2358] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:05:46 2022] POST /identity/v3/users => generated 299 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:05:46.366750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875fa2e-5e86-43cb-8521-5cb32c78b956 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:46.377756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875fa2e-5e86-43cb-8521-5cb32c78b956 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:46.383220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da277b6-202f-4343-ab38-91af82abf84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:46.383817 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da277b6-202f-4343-ab38-91af82abf84e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:46.384192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da277b6-202f-4343-ab38-91af82abf84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:46.384506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da277b6-202f-4343-ab38-91af82abf84e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:46.385196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da277b6-202f-4343-ab38-91af82abf84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:46.386762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da277b6-202f-4343-ab38-91af82abf84e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:46.399063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1175/2359] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:05:46.403303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e7a75-f181-41ef-ba85-7d2c8cfecef7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:46.413912 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e7a75-f181-41ef-ba85-7d2c8cfecef7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:46.418751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274aa9f-b09a-4336-9e51-42fa329ccdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:46.419391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274aa9f-b09a-4336-9e51-42fa329ccdc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:46.419670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274aa9f-b09a-4336-9e51-42fa329ccdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:46.419940 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274aa9f-b09a-4336-9e51-42fa329ccdc1 admin admin] PATH_INFO: `/v3/projects/bb68755464c649028d7429cf761f185d/users/a1b4bcd906174f0c9e539fcb50965214/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:46.435185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274aa9f-b09a-4336-9e51-42fa329ccdc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb68755464c649028d7429cf761f185d, user_id=a1b4bcd906174f0c9e539fcb50965214, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:46.437854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274aa9f-b09a-4336-9e51-42fa329ccdc1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:46.533234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1185/2360] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:05:46 2022] PUT /identity/v3/projects/bb68755464c649028d7429cf761f185d/users/a1b4bcd906174f0c9e539fcb50965214/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:46.537321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da277b6-202f-4343-ab38-91af82abf84e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:46.547377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da277b6-202f-4343-ab38-91af82abf84e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:46.604945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd5c38-5109-4d83-a41a-f4e33d115d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:46.605591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd5c38-5109-4d83-a41a-f4e33d115d88 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:46.605871 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd5c38-5109-4d83-a41a-f4e33d115d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:46.606454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd5c38-5109-4d83-a41a-f4e33d115d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:46.606870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd5c38-5109-4d83-a41a-f4e33d115d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:46.608478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd5c38-5109-4d83-a41a-f4e33d115d88 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:46.619476 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1176/2361] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:05:46.623666 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274aa9f-b09a-4336-9e51-42fa329ccdc1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:46.635549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274aa9f-b09a-4336-9e51-42fa329ccdc1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:46.640523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197264f-184a-4c4d-a4fa-971253fa6f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:46.641226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197264f-184a-4c4d-a4fa-971253fa6f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:46.641509 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197264f-184a-4c4d-a4fa-971253fa6f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:46.641786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197264f-184a-4c4d-a4fa-971253fa6f28 admin admin] PATH_INFO: `/v3/projects/bb68755464c649028d7429cf761f185d/users/a1b4bcd906174f0c9e539fcb50965214/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:46.643843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197264f-184a-4c4d-a4fa-971253fa6f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb68755464c649028d7429cf761f185d, user_id=a1b4bcd906174f0c9e539fcb50965214, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:46.646646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197264f-184a-4c4d-a4fa-971253fa6f28 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:46.657229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1186/2362] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:05:46 2022] PUT /identity/v3/projects/bb68755464c649028d7429cf761f185d/users/a1b4bcd906174f0c9e539fcb50965214/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:46.667831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3c2dd-76b3-4c61-83ac-ed0aa10056db None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:46.668220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3c2dd-76b3-4c61-83ac-ed0aa10056db None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:46.668504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3c2dd-76b3-4c61-83ac-ed0aa10056db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:46.701552 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-78d3c2dd-76b3-4c61-83ac-ed0aa10056db None None] MFA Rules not processed for user `a1b4bcd906174f0c9e539fcb50965214`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:46.702860 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-78d3c2dd-76b3-4c61-83ac-ed0aa10056db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:46.747045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:46.776208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1177/2363] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:46.787734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197264f-184a-4c4d-a4fa-971253fa6f28 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:46.790245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c197264f-184a-4c4d-a4fa-971253fa6f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:46.796699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:46.851253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197264f-184a-4c4d-a4fa-971253fa6f28 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:46.854936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce425936-1d9b-4618-8308-78bd4330f7b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:46.855547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce425936-1d9b-4618-8308-78bd4330f7b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:46.855825 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce425936-1d9b-4618-8308-78bd4330f7b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:46.856098 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce425936-1d9b-4618-8308-78bd4330f7b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:46.866533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce425936-1d9b-4618-8308-78bd4330f7b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:46.867966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce425936-1d9b-4618-8308-78bd4330f7b0 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:46.878575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1187/2364] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:48.011195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3c2dd-76b3-4c61-83ac-ed0aa10056db None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:48.012902 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-78d3c2dd-76b3-4c61-83ac-ed0aa10056db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:48.024939 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3c2dd-76b3-4c61-83ac-ed0aa10056db None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:48.028883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840fa18-6cc1-48f2-888b-2c9f4fe0be07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:48.029532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840fa18-6cc1-48f2-888b-2c9f4fe0be07 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:48.029815 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840fa18-6cc1-48f2-888b-2c9f4fe0be07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:48.030087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840fa18-6cc1-48f2-888b-2c9f4fe0be07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:48.103229 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840fa18-6cc1-48f2-888b-2c9f4fe0be07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:48.104651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840fa18-6cc1-48f2-888b-2c9f4fe0be07 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:48.114035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1178/2365] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:51.408019 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10d9e1-7565-4dd8-aa6c-a53329a7381b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:51.408533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10d9e1-7565-4dd8-aa6c-a53329a7381b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:51.408629 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10d9e1-7565-4dd8-aa6c-a53329a7381b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:51.429004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-7a10d9e1-7565-4dd8-aa6c-a53329a7381b None None] MFA Rules not processed for user `ee81d8d9cf994df68a15afe5cb9673c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:51.430238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-7a10d9e1-7565-4dd8-aa6c-a53329a7381b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:51.486387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1188/2366] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:52.113626 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840fa18-6cc1-48f2-888b-2c9f4fe0be07 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:52.115039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3840fa18-6cc1-48f2-888b-2c9f4fe0be07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:52.171583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840fa18-6cc1-48f2-888b-2c9f4fe0be07 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:52.175086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b717ab0-0de4-4add-a8c5-6c2d43013676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:52.175682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b717ab0-0de4-4add-a8c5-6c2d43013676 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:52.175957 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b717ab0-0de4-4add-a8c5-6c2d43013676 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:52.176224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b717ab0-0de4-4add-a8c5-6c2d43013676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:52.178400 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3b717ab0-0de4-4add-a8c5-6c2d43013676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:52.187932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b717ab0-0de4-4add-a8c5-6c2d43013676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:52.189299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b717ab0-0de4-4add-a8c5-6c2d43013676 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:52.197445 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1179/2367] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:53.042484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10d9e1-7565-4dd8-aa6c-a53329a7381b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:53.048457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10d9e1-7565-4dd8-aa6c-a53329a7381b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:53.052245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba5b0d-ebed-4e39-a97a-e62169cff90a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:53.052750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba5b0d-ebed-4e39-a97a-e62169cff90a service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:53.053225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba5b0d-ebed-4e39-a97a-e62169cff90a service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:53.053607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba5b0d-ebed-4e39-a97a-e62169cff90a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:53.122205 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba5b0d-ebed-4e39-a97a-e62169cff90a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:53.123646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba5b0d-ebed-4e39-a97a-e62169cff90a service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:53.133384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1189/2368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:57.374021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b717ab0-0de4-4add-a8c5-6c2d43013676 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:57.458876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b717ab0-0de4-4add-a8c5-6c2d43013676 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:57.465353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:57.466258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:57.466741 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:57.467607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] PATH_INFO: `/v3/users/b966c60cd2f942d78b644ff6598f38d8` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:57.468815 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b966c60cd2f942d78b644ff6598f38d8)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:57.470823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:57.498866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:57.720221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b966c60cd2f942d78b644ff6598f38d8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:57.720878 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b966c60cd2f942d78b644ff6598f38d8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:57.721545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b966c60cd2f942d78b644ff6598f38d8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:57.796581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b966c60cd2f942d78b644ff6598f38d8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:57.809458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b966c60cd2f942d78b644ff6598f38d8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:57.822049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1180/2369] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:05:57 2022] DELETE /identity/v3/users/b966c60cd2f942d78b644ff6598f38d8 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:57.826676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005a815-ee89-41f9-afb5-0a956b3b152d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:57.827165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005a815-ee89-41f9-afb5-0a956b3b152d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:57.827259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005a815-ee89-41f9-afb5-0a956b3b152d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:57.852113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3005a815-ee89-41f9-afb5-0a956b3b152d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:05:57.853736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3005a815-ee89-41f9-afb5-0a956b3b152d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:57.901002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:57.914788 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1190/2370] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:05:57.924677 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:57.928132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:57.983937 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37083648-7ea7-40bf-a9c7-066905c437cd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:57.987591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771c7d6-41d9-4360-be81-fb39468a38c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:57.988193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771c7d6-41d9-4360-be81-fb39468a38c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:57.988472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771c7d6-41d9-4360-be81-fb39468a38c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:57.988743 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771c7d6-41d9-4360-be81-fb39468a38c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:57.997823 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771c7d6-41d9-4360-be81-fb39468a38c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:57.999230 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0771c7d6-41d9-4360-be81-fb39468a38c4 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:58.009370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1181/2371] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:05:58.457086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005a815-ee89-41f9-afb5-0a956b3b152d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:58.458563 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3005a815-ee89-41f9-afb5-0a956b3b152d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:58.471764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005a815-ee89-41f9-afb5-0a956b3b152d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:58.477113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:58.478568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:58.478890 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:58.479167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] PATH_INFO: `/v3/projects/1a1fc6a4bd7a410597921cc0a74b0d18` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:58.480534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a1fc6a4bd7a410597921cc0a74b0d18)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:58.482447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:58.610028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a1fc6a4bd7a410597921cc0a74b0d18', 'request_id': 'req-59c5ae92-4e85-43e4-9292-d97ba2d270b3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:58.617571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a1fc6a4bd7a410597921cc0a74b0d18', 'request_id': 'req-59c5ae92-4e85-43e4-9292-d97ba2d270b3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:58.624201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a1fc6a4bd7a410597921cc0a74b0d18', 'request_id': 'req-59c5ae92-4e85-43e4-9292-d97ba2d270b3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:58.718444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] The token cache is being invalidate because project 1a1fc6a4bd7a410597921cc0a74b0d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:05:58.718812 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:58.719592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:58.721557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1191/2372] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:05:58 2022] DELETE /identity/v3/projects/1a1fc6a4bd7a410597921cc0a74b0d18 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:58.725818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771c7d6-41d9-4360-be81-fb39468a38c4 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:58.741692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771c7d6-41d9-4360-be81-fb39468a38c4 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:58.813419 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:58.814220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:58.814504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:58.814777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] PATH_INFO: `/v3/users/b59e55550a1644e38db5fe2fe1945b9f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:58.815914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b59e55550a1644e38db5fe2fe1945b9f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:58.822649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:59.016911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:05:59.021723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b59e55550a1644e38db5fe2fe1945b9f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:59.022388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b59e55550a1644e38db5fe2fe1945b9f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:59.023138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b59e55550a1644e38db5fe2fe1945b9f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:59.081539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b59e55550a1644e38db5fe2fe1945b9f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:59.094656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b59e55550a1644e38db5fe2fe1945b9f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:59.108070 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1182/2373] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:05:58 2022] DELETE /identity/v3/users/b59e55550a1644e38db5fe2fe1945b9f => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:59.626191 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:59.630506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:59.641437 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5ae92-4e85-43e4-9292-d97ba2d270b3 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:05:59.702649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:05:59.703202 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:05:59.703479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:05:59.703750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] PATH_INFO: `/v3/projects/a34d40b7cc064e59a8ae2171d0b4c812` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:05:59.704849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a34d40b7cc064e59a8ae2171d0b4c812)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:05:59.706793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:05:59.826460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a34d40b7cc064e59a8ae2171d0b4c812', 'request_id': 'req-382f5d15-084d-48c1-8a57-48c5712b0253'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:59.832917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a34d40b7cc064e59a8ae2171d0b4c812', 'request_id': 'req-382f5d15-084d-48c1-8a57-48c5712b0253'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:59.848015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a34d40b7cc064e59a8ae2171d0b4c812', 'request_id': 'req-382f5d15-084d-48c1-8a57-48c5712b0253'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:59.922063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] The token cache is being invalidate because project a34d40b7cc064e59a8ae2171d0b4c812 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:05:59.922422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:59.923095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:05:59.924897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1192/2374] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:05:59 2022] DELETE /identity/v3/projects/a34d40b7cc064e59a8ae2171d0b4c812 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:05:59.929173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:05:59.931448 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:05:59.945725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaba0b5-7260-4d63-b1fb-b0989f79818b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:00.027635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600e663-776a-43f7-b95c-f7f3cf65dbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:00.028398 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600e663-776a-43f7-b95c-f7f3cf65dbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:00.028727 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600e663-776a-43f7-b95c-f7f3cf65dbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:00.029107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600e663-776a-43f7-b95c-f7f3cf65dbd2 admin admin] PATH_INFO: `/v3/users/a1b4bcd906174f0c9e539fcb50965214` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:00.030566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600e663-776a-43f7-b95c-f7f3cf65dbd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b4bcd906174f0c9e539fcb50965214)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:00.032725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600e663-776a-43f7-b95c-f7f3cf65dbd2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:00.204369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4600e663-776a-43f7-b95c-f7f3cf65dbd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b4bcd906174f0c9e539fcb50965214'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:00.205032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4600e663-776a-43f7-b95c-f7f3cf65dbd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b4bcd906174f0c9e539fcb50965214'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:00.205675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4600e663-776a-43f7-b95c-f7f3cf65dbd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b4bcd906174f0c9e539fcb50965214'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:00.276281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4600e663-776a-43f7-b95c-f7f3cf65dbd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b4bcd906174f0c9e539fcb50965214'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:00.288656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4600e663-776a-43f7-b95c-f7f3cf65dbd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b4bcd906174f0c9e539fcb50965214'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:00.300931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1183/2375] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:05:59 2022] DELETE /identity/v3/users/a1b4bcd906174f0c9e539fcb50965214 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:06:00.757762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:00.759813 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:00.769670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:00.772575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f5d15-084d-48c1-8a57-48c5712b0253 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:00.831242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:00.831884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:00.832161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:00.832436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] PATH_INFO: `/v3/projects/bb68755464c649028d7429cf761f185d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:00.833570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb68755464c649028d7429cf761f185d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:00.835514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:00.934764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb68755464c649028d7429cf761f185d', 'request_id': 'req-b611650f-7ae7-4b0e-aed5-1874c46b66ad'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:00.940384 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb68755464c649028d7429cf761f185d', 'request_id': 'req-b611650f-7ae7-4b0e-aed5-1874c46b66ad'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:00.946542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb68755464c649028d7429cf761f185d', 'request_id': 'req-b611650f-7ae7-4b0e-aed5-1874c46b66ad'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:01.024103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] The token cache is being invalidate because project bb68755464c649028d7429cf761f185d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:06:01.024538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:01.025351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b611650f-7ae7-4b0e-aed5-1874c46b66ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:01.026820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1193/2376] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:06:00 2022] DELETE /identity/v3/projects/bb68755464c649028d7429cf761f185d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:01.032694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e290e4f-2e37-4627-b1f8-04087ab071ca None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:01.033254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e290e4f-2e37-4627-b1f8-04087ab071ca None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:01.033254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e290e4f-2e37-4627-b1f8-04087ab071ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:01.058461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8e290e4f-2e37-4627-b1f8-04087ab071ca None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:01.059653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8e290e4f-2e37-4627-b1f8-04087ab071ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:01.093454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:01.122371 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1184/2377] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:01.132066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b2c20-3514-47c8-9b86-2ff7c1612988 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:01.132349 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b2c20-3514-47c8-9b86-2ff7c1612988 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:01.132623 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b2c20-3514-47c8-9b86-2ff7c1612988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:01.152840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0b2b2c20-3514-47c8-9b86-2ff7c1612988 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:01.157192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0b2b2c20-3514-47c8-9b86-2ff7c1612988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:01.162179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1194/2378] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:01.167188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e290e4f-2e37-4627-b1f8-04087ab071ca None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:01.177221 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e290e4f-2e37-4627-b1f8-04087ab071ca None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:01.182690 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1606a-bf11-4865-93a3-91312a4f2634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:01.183302 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1606a-bf11-4865-93a3-91312a4f2634 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:01.183589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1606a-bf11-4865-93a3-91312a4f2634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:01.183869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1606a-bf11-4865-93a3-91312a4f2634 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:01.184523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1606a-bf11-4865-93a3-91312a4f2634 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:06:01.184829 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1606a-bf11-4865-93a3-91312a4f2634 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:01.186451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1606a-bf11-4865-93a3-91312a4f2634 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:01.199194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1185/2379] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:06:01.203781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b2c20-3514-47c8-9b86-2ff7c1612988 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:01.214033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b2c20-3514-47c8-9b86-2ff7c1612988 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:01.219117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135653e8-3862-40bd-a730-8c710742bc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:01.219717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135653e8-3862-40bd-a730-8c710742bc95 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:01.220026 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135653e8-3862-40bd-a730-8c710742bc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:01.220301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135653e8-3862-40bd-a730-8c710742bc95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:01.220927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135653e8-3862-40bd-a730-8c710742bc95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:01.222866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135653e8-3862-40bd-a730-8c710742bc95 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:01.289145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1195/2380] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:06:01 2022] POST /identity/v3/projects => generated 407 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:01.294103 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1606a-bf11-4865-93a3-91312a4f2634 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:01.304203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1606a-bf11-4865-93a3-91312a4f2634 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:01.359309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ee662-b5a1-4166-bb18-2cb22c8f8341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:01.359930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ee662-b5a1-4166-bb18-2cb22c8f8341 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:01.360213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ee662-b5a1-4166-bb18-2cb22c8f8341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:01.360485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ee662-b5a1-4166-bb18-2cb22c8f8341 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:01.361145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ee662-b5a1-4166-bb18-2cb22c8f8341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:01.363025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ee662-b5a1-4166-bb18-2cb22c8f8341 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:01.436427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1186/2381] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:06:01 2022] POST /identity/v3/users => generated 304 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:01.440787 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135653e8-3862-40bd-a730-8c710742bc95 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:01.452007 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135653e8-3862-40bd-a730-8c710742bc95 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:01.456633 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182315b-e67c-4f5e-af14-0808f298f347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:01.457309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182315b-e67c-4f5e-af14-0808f298f347 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:01.457575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182315b-e67c-4f5e-af14-0808f298f347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:01.457847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182315b-e67c-4f5e-af14-0808f298f347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:01.458584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182315b-e67c-4f5e-af14-0808f298f347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:01.460245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182315b-e67c-4f5e-af14-0808f298f347 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:01.470721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1196/2382] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:01.474603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ee662-b5a1-4166-bb18-2cb22c8f8341 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:01.484777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ee662-b5a1-4166-bb18-2cb22c8f8341 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:01.489451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3f47d-a306-4ce1-912d-7a8edec7a1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:01.490043 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3f47d-a306-4ce1-912d-7a8edec7a1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:01.490322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3f47d-a306-4ce1-912d-7a8edec7a1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:01.490591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3f47d-a306-4ce1-912d-7a8edec7a1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:01.491225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3f47d-a306-4ce1-912d-7a8edec7a1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:01.492773 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3f47d-a306-4ce1-912d-7a8edec7a1e9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:01.504025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1187/2383] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:01.507984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182315b-e67c-4f5e-af14-0808f298f347 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:01.519523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182315b-e67c-4f5e-af14-0808f298f347 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:01.524172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c137b4-8cf0-43c8-bbd4-1a822c859b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:01.524810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c137b4-8cf0-43c8-bbd4-1a822c859b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:01.525165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c137b4-8cf0-43c8-bbd4-1a822c859b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:01.525440 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c137b4-8cf0-43c8-bbd4-1a822c859b82 admin admin] PATH_INFO: `/v3/projects/2f5b190767b04e628abccf9adbda3f9b/users/9ecd98f374d24c67a34be4d2335515a9/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:01.540074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c137b4-8cf0-43c8-bbd4-1a822c859b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5b190767b04e628abccf9adbda3f9b, user_id=9ecd98f374d24c67a34be4d2335515a9, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:01.543349 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c137b4-8cf0-43c8-bbd4-1a822c859b82 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:01.608256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1197/2384] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:01 2022] PUT /identity/v3/projects/2f5b190767b04e628abccf9adbda3f9b/users/9ecd98f374d24c67a34be4d2335515a9/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:01.620364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ad661-693f-4773-9a1f-5f559b670d0c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:01.620681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ad661-693f-4773-9a1f-5f559b670d0c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:01.620956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ad661-693f-4773-9a1f-5f559b670d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:01.648063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ad4ad661-693f-4773-9a1f-5f559b670d0c None None] MFA Rules not processed for user `9ecd98f374d24c67a34be4d2335515a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:01.649301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ad4ad661-693f-4773-9a1f-5f559b670d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:01.718558 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1188/2385] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:01 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:01.722657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c137b4-8cf0-43c8-bbd4-1a822c859b82 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:01.758833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:01.795034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c137b4-8cf0-43c8-bbd4-1a822c859b82 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:01.799751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f57c8c-2ee6-47d3-9cc8-b4831cd9ed54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:01.800406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f57c8c-2ee6-47d3-9cc8-b4831cd9ed54 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:01.800902 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f57c8c-2ee6-47d3-9cc8-b4831cd9ed54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:01.801527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f57c8c-2ee6-47d3-9cc8-b4831cd9ed54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:01.802599 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f57c8c-2ee6-47d3-9cc8-b4831cd9ed54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:01.804543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f57c8c-2ee6-47d3-9cc8-b4831cd9ed54 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:01.888586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1198/2386] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:06:01 2022] POST /identity/v3/projects => generated 405 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:01.892772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ad661-693f-4773-9a1f-5f559b670d0c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:01.903876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ad661-693f-4773-9a1f-5f559b670d0c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:01.934151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:01.960157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9dfb0-ff49-4705-9c90-1854eb8a4f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:01.962425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9dfb0-ff49-4705-9c90-1854eb8a4f7f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:01.962425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9dfb0-ff49-4705-9c90-1854eb8a4f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:01.962425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9dfb0-ff49-4705-9c90-1854eb8a4f7f admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:01.962425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9dfb0-ff49-4705-9c90-1854eb8a4f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:01.964502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9dfb0-ff49-4705-9c90-1854eb8a4f7f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:02.053613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1189/2387] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:06:01 2022] POST /identity/v3/users => generated 309 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:02.057790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f57c8c-2ee6-47d3-9cc8-b4831cd9ed54 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:02.067887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f57c8c-2ee6-47d3-9cc8-b4831cd9ed54 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:02.072827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85fb14-2b97-4c55-88d1-dbc7a6ed2646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:02.073526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85fb14-2b97-4c55-88d1-dbc7a6ed2646 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:02.073808 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85fb14-2b97-4c55-88d1-dbc7a6ed2646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:02.074076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85fb14-2b97-4c55-88d1-dbc7a6ed2646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:02.076654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85fb14-2b97-4c55-88d1-dbc7a6ed2646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:02.076654 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85fb14-2b97-4c55-88d1-dbc7a6ed2646 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:02.086318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1199/2388] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:06:02.090906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9dfb0-ff49-4705-9c90-1854eb8a4f7f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:02.101416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9dfb0-ff49-4705-9c90-1854eb8a4f7f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:02.107376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc040af1-2349-4803-be0b-644d76e16a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:02.108018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc040af1-2349-4803-be0b-644d76e16a7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:02.108299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc040af1-2349-4803-be0b-644d76e16a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:02.108573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc040af1-2349-4803-be0b-644d76e16a7f admin admin] PATH_INFO: `/v3/projects/0f9aac2e9f90482298aeb7e711485932/users/57b39ae109c8411c823bbb7ff1b777d3/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:02.123178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc040af1-2349-4803-be0b-644d76e16a7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9aac2e9f90482298aeb7e711485932, user_id=57b39ae109c8411c823bbb7ff1b777d3, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:02.125885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc040af1-2349-4803-be0b-644d76e16a7f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:02.167855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1190/2389] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:02 2022] PUT /identity/v3/projects/0f9aac2e9f90482298aeb7e711485932/users/57b39ae109c8411c823bbb7ff1b777d3/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:02.171885 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85fb14-2b97-4c55-88d1-dbc7a6ed2646 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:02.181779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85fb14-2b97-4c55-88d1-dbc7a6ed2646 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:02.240636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfba04a-c2f9-49b5-90e2-5370f24a094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:02.241268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfba04a-c2f9-49b5-90e2-5370f24a094d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:02.241548 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfba04a-c2f9-49b5-90e2-5370f24a094d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:02.241819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfba04a-c2f9-49b5-90e2-5370f24a094d admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:02.242600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfba04a-c2f9-49b5-90e2-5370f24a094d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:02.244354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfba04a-c2f9-49b5-90e2-5370f24a094d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:02.257570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1200/2390] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:02.258728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc040af1-2349-4803-be0b-644d76e16a7f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:02.270072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc040af1-2349-4803-be0b-644d76e16a7f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:02.275014 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7d399-9870-4dc8-80f7-1fa535b020f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:02.276562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7d399-9870-4dc8-80f7-1fa535b020f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:02.276898 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7d399-9870-4dc8-80f7-1fa535b020f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:02.277213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7d399-9870-4dc8-80f7-1fa535b020f8 admin admin] PATH_INFO: `/v3/projects/0f9aac2e9f90482298aeb7e711485932/users/57b39ae109c8411c823bbb7ff1b777d3/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:02.279142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7d399-9870-4dc8-80f7-1fa535b020f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9aac2e9f90482298aeb7e711485932, user_id=57b39ae109c8411c823bbb7ff1b777d3, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:02.281808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7d399-9870-4dc8-80f7-1fa535b020f8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:02.292792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1191/2391] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:02 2022] PUT /identity/v3/projects/0f9aac2e9f90482298aeb7e711485932/users/57b39ae109c8411c823bbb7ff1b777d3/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:02.304018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983fa8d-27b9-4695-83da-1283fe7b88e3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:02.304236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983fa8d-27b9-4695-83da-1283fe7b88e3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:02.304511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983fa8d-27b9-4695-83da-1283fe7b88e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:02.333348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9983fa8d-27b9-4695-83da-1283fe7b88e3 None None] MFA Rules not processed for user `57b39ae109c8411c823bbb7ff1b777d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:02.334569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9983fa8d-27b9-4695-83da-1283fe7b88e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:02.407712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1201/2392] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:02.418732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7d399-9870-4dc8-80f7-1fa535b020f8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:02.429936 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4ab7d399-9870-4dc8-80f7-1fa535b020f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:02.480721 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7d399-9870-4dc8-80f7-1fa535b020f8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:02.484538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f72fda-1455-432b-a3ed-93eb6764a106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:02.485185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72fda-1455-432b-a3ed-93eb6764a106 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:02.485470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72fda-1455-432b-a3ed-93eb6764a106 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:02.485748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72fda-1455-432b-a3ed-93eb6764a106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:02.495266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f72fda-1455-432b-a3ed-93eb6764a106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:02.496689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f72fda-1455-432b-a3ed-93eb6764a106 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:02.506744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1192/2393] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:02 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:02.856721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983fa8d-27b9-4695-83da-1283fe7b88e3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:02.863526 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983fa8d-27b9-4695-83da-1283fe7b88e3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:02.867090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ea64c-f4e7-45f6-9b69-39d22e2acff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:02.867831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ea64c-f4e7-45f6-9b69-39d22e2acff2 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:02.868198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ea64c-f4e7-45f6-9b69-39d22e2acff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:02.868845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ea64c-f4e7-45f6-9b69-39d22e2acff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:02.896760 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:02.930917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ea64c-f4e7-45f6-9b69-39d22e2acff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:02.932873 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ea64c-f4e7-45f6-9b69-39d22e2acff2 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:02.942092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1202/2394] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:02 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:12.251092 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f72fda-1455-432b-a3ed-93eb6764a106 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:12.252536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f7f72fda-1455-432b-a3ed-93eb6764a106 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:12.312899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f72fda-1455-432b-a3ed-93eb6764a106 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:12.316391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca7b04-b436-42c2-a656-3cbc6c85b6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:12.317005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca7b04-b436-42c2-a656-3cbc6c85b6ce service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:12.317330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca7b04-b436-42c2-a656-3cbc6c85b6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:12.317598 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca7b04-b436-42c2-a656-3cbc6c85b6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:12.319350 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a2ca7b04-b436-42c2-a656-3cbc6c85b6ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:12.365292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:12.387457 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca7b04-b436-42c2-a656-3cbc6c85b6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:12.389248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca7b04-b436-42c2-a656-3cbc6c85b6ce service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:12.399420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1193/2395] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:12.859343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ea64c-f4e7-45f6-9b69-39d22e2acff2 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:12.881658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ea64c-f4e7-45f6-9b69-39d22e2acff2 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:12.886413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:12.887241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:12.887529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:12.887805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] PATH_INFO: `/v3/users/9ecd98f374d24c67a34be4d2335515a9` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:12.888950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ecd98f374d24c67a34be4d2335515a9)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:12.890904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:13.113849 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ecd98f374d24c67a34be4d2335515a9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:13.120949 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ecd98f374d24c67a34be4d2335515a9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:13.187677 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ecd98f374d24c67a34be4d2335515a9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:13.188714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ecd98f374d24c67a34be4d2335515a9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:13.204000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ecd98f374d24c67a34be4d2335515a9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:13.210150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1203/2396] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:06:12 2022] DELETE /identity/v3/users/9ecd98f374d24c67a34be4d2335515a9 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:13.215420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032a77e-c807-4914-a9c9-ecd331fe161d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:13.215926 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032a77e-c807-4914-a9c9-ecd331fe161d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:13.216356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032a77e-c807-4914-a9c9-ecd331fe161d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:13.236470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e032a77e-c807-4914-a9c9-ecd331fe161d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:13.237718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e032a77e-c807-4914-a9c9-ecd331fe161d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:13.300462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1194/2397] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:13.317468 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:13.319084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:13.412645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e688a-5fd9-4ee7-aa80-c5f013a2720e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:13.416573 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2af63-78af-41fa-ad04-12ffc8fc7791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:13.417250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2af63-78af-41fa-ad04-12ffc8fc7791 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:13.417738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2af63-78af-41fa-ad04-12ffc8fc7791 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:13.418060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2af63-78af-41fa-ad04-12ffc8fc7791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:13.437492 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2af63-78af-41fa-ad04-12ffc8fc7791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:13.440423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2af63-78af-41fa-ad04-12ffc8fc7791 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:13.456345 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1204/2398] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:14.303968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032a77e-c807-4914-a9c9-ecd331fe161d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:14.307025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e032a77e-c807-4914-a9c9-ecd331fe161d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:14.330153 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032a77e-c807-4914-a9c9-ecd331fe161d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:14.340893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:14.344238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:14.344238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:14.344238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] PATH_INFO: `/v3/projects/2f5b190767b04e628abccf9adbda3f9b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:14.345158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f5b190767b04e628abccf9adbda3f9b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:14.347782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:14.472391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f5b190767b04e628abccf9adbda3f9b', 'request_id': 'req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:14.479265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f5b190767b04e628abccf9adbda3f9b', 'request_id': 'req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:14.487749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f5b190767b04e628abccf9adbda3f9b', 'request_id': 'req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:14.576969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] The token cache is being invalidate because project 2f5b190767b04e628abccf9adbda3f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:06:14.578038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:14.578990 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:14.603766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1195/2399] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:06:14 2022] DELETE /identity/v3/projects/2f5b190767b04e628abccf9adbda3f9b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:14.604073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2af63-78af-41fa-ad04-12ffc8fc7791 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:14.616237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2af63-78af-41fa-ad04-12ffc8fc7791 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:14.680550 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:14.706190 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fc481-5324-45e9-9c3d-7fc28de27754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:14.707282 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fc481-5324-45e9-9c3d-7fc28de27754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:14.708594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fc481-5324-45e9-9c3d-7fc28de27754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:14.709160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fc481-5324-45e9-9c3d-7fc28de27754 admin admin] PATH_INFO: `/v3/users/57b39ae109c8411c823bbb7ff1b777d3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:14.712105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fc481-5324-45e9-9c3d-7fc28de27754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b39ae109c8411c823bbb7ff1b777d3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:14.715498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fc481-5324-45e9-9c3d-7fc28de27754 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:14.907324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d43fc481-5324-45e9-9c3d-7fc28de27754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b39ae109c8411c823bbb7ff1b777d3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:14.915100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d43fc481-5324-45e9-9c3d-7fc28de27754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b39ae109c8411c823bbb7ff1b777d3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:14.974023 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d43fc481-5324-45e9-9c3d-7fc28de27754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b39ae109c8411c823bbb7ff1b777d3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:14.977878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d43fc481-5324-45e9-9c3d-7fc28de27754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b39ae109c8411c823bbb7ff1b777d3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:14.994273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d43fc481-5324-45e9-9c3d-7fc28de27754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b39ae109c8411c823bbb7ff1b777d3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:15.006348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1205/2400] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:06:14 2022] DELETE /identity/v3/users/57b39ae109c8411c823bbb7ff1b777d3 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:15.846140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:15.847661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:15.865575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a837f3e-6cfd-45f0-92bc-c93c7ca726b3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:16.006311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:16.007193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:16.007920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:16.008355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] PATH_INFO: `/v3/projects/0f9aac2e9f90482298aeb7e711485932` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:16.009920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f9aac2e9f90482298aeb7e711485932)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:16.013838 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:16.018401 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:16.122190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f9aac2e9f90482298aeb7e711485932', 'request_id': 'req-898fd89d-3b55-4ee1-b7f4-7a686357346e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:16.129100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f9aac2e9f90482298aeb7e711485932', 'request_id': 'req-898fd89d-3b55-4ee1-b7f4-7a686357346e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:16.137768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f9aac2e9f90482298aeb7e711485932', 'request_id': 'req-898fd89d-3b55-4ee1-b7f4-7a686357346e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:16.242729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] The token cache is being invalidate because project 0f9aac2e9f90482298aeb7e711485932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:06:16.242729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:16.243431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-898fd89d-3b55-4ee1-b7f4-7a686357346e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:16.245982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1196/2401] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:06:15 2022] DELETE /identity/v3/projects/0f9aac2e9f90482298aeb7e711485932 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:16.250802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615eae4-b78b-42ff-a5bd-fb25d6997b30 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:16.252088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615eae4-b78b-42ff-a5bd-fb25d6997b30 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:16.252579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615eae4-b78b-42ff-a5bd-fb25d6997b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:16.291041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c615eae4-b78b-42ff-a5bd-fb25d6997b30 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:16.294057 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c615eae4-b78b-42ff-a5bd-fb25d6997b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:16.405489 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1206/2402] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:16.411329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957125-c5a1-4f80-bf91-6f7b1a212bd8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:16.411821 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957125-c5a1-4f80-bf91-6f7b1a212bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:16.412095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957125-c5a1-4f80-bf91-6f7b1a212bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:16.435801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6c957125-c5a1-4f80-bf91-6f7b1a212bd8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:16.437244 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6c957125-c5a1-4f80-bf91-6f7b1a212bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:16.452155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615eae4-b78b-42ff-a5bd-fb25d6997b30 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:16.452971 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1197/2403] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:16.464969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615eae4-b78b-42ff-a5bd-fb25d6997b30 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:16.471560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819c46b-d2cc-4daa-a2e1-5ae0f9d18ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:16.472310 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819c46b-d2cc-4daa-a2e1-5ae0f9d18ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:16.472775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819c46b-d2cc-4daa-a2e1-5ae0f9d18ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:16.473324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819c46b-d2cc-4daa-a2e1-5ae0f9d18ad0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:16.474105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819c46b-d2cc-4daa-a2e1-5ae0f9d18ad0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:06:16.474527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819c46b-d2cc-4daa-a2e1-5ae0f9d18ad0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:16.476657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819c46b-d2cc-4daa-a2e1-5ae0f9d18ad0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:16.499827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c957125-c5a1-4f80-bf91-6f7b1a212bd8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:16.500503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1207/2404] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:06:16.509552 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c957125-c5a1-4f80-bf91-6f7b1a212bd8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:16.515692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e2bbe-30d8-418d-bcba-2ef72b113756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:16.516375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e2bbe-30d8-418d-bcba-2ef72b113756 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:16.517679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e2bbe-30d8-418d-bcba-2ef72b113756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:16.517679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e2bbe-30d8-418d-bcba-2ef72b113756 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:16.517679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e2bbe-30d8-418d-bcba-2ef72b113756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:16.519569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e2bbe-30d8-418d-bcba-2ef72b113756 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:16.600628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1198/2405] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:06:16 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:06:16.605452 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819c46b-d2cc-4daa-a2e1-5ae0f9d18ad0 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:16.616184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819c46b-d2cc-4daa-a2e1-5ae0f9d18ad0 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:16.682065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:16.697790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffb9c3-2af3-43ab-8956-86653d46c4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:16.703050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffb9c3-2af3-43ab-8956-86653d46c4df admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:16.703050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffb9c3-2af3-43ab-8956-86653d46c4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:16.703050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffb9c3-2af3-43ab-8956-86653d46c4df admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:16.703050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ffb9c3-2af3-43ab-8956-86653d46c4df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:16.703050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ffb9c3-2af3-43ab-8956-86653d46c4df admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:16.787979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1208/2406] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:06:16 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:16.792037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e2bbe-30d8-418d-bcba-2ef72b113756 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:16.804624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e2bbe-30d8-418d-bcba-2ef72b113756 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:16.816197 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21374d46-7f89-4e98-a4c3-12a071a3280b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:16.817460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21374d46-7f89-4e98-a4c3-12a071a3280b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:16.817929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21374d46-7f89-4e98-a4c3-12a071a3280b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:16.818293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21374d46-7f89-4e98-a4c3-12a071a3280b admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:16.819144 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21374d46-7f89-4e98-a4c3-12a071a3280b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:16.821086 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21374d46-7f89-4e98-a4c3-12a071a3280b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:16.856511 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1199/2407] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:16.860727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffb9c3-2af3-43ab-8956-86653d46c4df admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:16.880127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ffb9c3-2af3-43ab-8956-86653d46c4df admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:16.885257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d3a92-b21b-4567-b115-88cf705794ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:16.885993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d3a92-b21b-4567-b115-88cf705794ce admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:16.886556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d3a92-b21b-4567-b115-88cf705794ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:16.886899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d3a92-b21b-4567-b115-88cf705794ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:16.887664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d3a92-b21b-4567-b115-88cf705794ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:16.889371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d3a92-b21b-4567-b115-88cf705794ce admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:16.901510 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1209/2408] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:16.908870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21374d46-7f89-4e98-a4c3-12a071a3280b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:16.920516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21374d46-7f89-4e98-a4c3-12a071a3280b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:16.925405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc75ab6-4af7-4a6f-b033-1f1445ca2274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:16.926166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75ab6-4af7-4a6f-b033-1f1445ca2274 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:16.926519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75ab6-4af7-4a6f-b033-1f1445ca2274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:16.926860 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75ab6-4af7-4a6f-b033-1f1445ca2274 admin admin] PATH_INFO: `/v3/projects/9784c47469514770908039754ceace65/users/bc30188074b04cda9f17003b47918994/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:16.945077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc75ab6-4af7-4a6f-b033-1f1445ca2274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9784c47469514770908039754ceace65, user_id=bc30188074b04cda9f17003b47918994, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:16.947909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc75ab6-4af7-4a6f-b033-1f1445ca2274 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:17.037520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1200/2409] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:16 2022] PUT /identity/v3/projects/9784c47469514770908039754ceace65/users/bc30188074b04cda9f17003b47918994/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:17.049203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6b51-b02d-4be6-8b5b-3e5c2c7f7c42 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:17.049506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6b51-b02d-4be6-8b5b-3e5c2c7f7c42 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:17.049774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6b51-b02d-4be6-8b5b-3e5c2c7f7c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:17.082905 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-466e6b51-b02d-4be6-8b5b-3e5c2c7f7c42 None None] MFA Rules not processed for user `bc30188074b04cda9f17003b47918994`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:17.084562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-466e6b51-b02d-4be6-8b5b-3e5c2c7f7c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:17.212706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1210/2410] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:17.226404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc75ab6-4af7-4a6f-b033-1f1445ca2274 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:17.228408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-dbc75ab6-4af7-4a6f-b033-1f1445ca2274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:17.297467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc75ab6-4af7-4a6f-b033-1f1445ca2274 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:17.301273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbd8fa-b7b1-4027-8c66-2c79643dc79e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:17.301891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbd8fa-b7b1-4027-8c66-2c79643dc79e service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:17.302172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbd8fa-b7b1-4027-8c66-2c79643dc79e service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:17.302451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbd8fa-b7b1-4027-8c66-2c79643dc79e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:17.311363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbd8fa-b7b1-4027-8c66-2c79643dc79e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:17.312851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbd8fa-b7b1-4027-8c66-2c79643dc79e service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:17.322128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1201/2411] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:17.342995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e6b51-b02d-4be6-8b5b-3e5c2c7f7c42 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:17.394897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbd8fa-b7b1-4027-8c66-2c79643dc79e service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:17.396981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4afbd8fa-b7b1-4027-8c66-2c79643dc79e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:17.438779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e6b51-b02d-4be6-8b5b-3e5c2c7f7c42 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:17.443818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6950562-bb91-483a-bea4-bfe1d1be4a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:17.444809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6950562-bb91-483a-bea4-bfe1d1be4a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:17.445210 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6950562-bb91-483a-bea4-bfe1d1be4a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:17.445863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6950562-bb91-483a-bea4-bfe1d1be4a8c admin admin] PATH_INFO: `/v3/users/bc30188074b04cda9f17003b47918994` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:17.447142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6950562-bb91-483a-bea4-bfe1d1be4a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc30188074b04cda9f17003b47918994)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:17.449743 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6950562-bb91-483a-bea4-bfe1d1be4a8c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:17.456682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:17.488806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbd8fa-b7b1-4027-8c66-2c79643dc79e service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:17.493446 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:17.497182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:17.497182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:17.497182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] PATH_INFO: `/v3/users/d6c2fc568ee74d35bba01c6ce294edeb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:17.497182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6c2fc568ee74d35bba01c6ce294edeb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:17.498475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:17.662171 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e6950562-bb91-483a-bea4-bfe1d1be4a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc30188074b04cda9f17003b47918994'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:17.669596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e6950562-bb91-483a-bea4-bfe1d1be4a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc30188074b04cda9f17003b47918994'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:17.672151 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:17.725278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6c2fc568ee74d35bba01c6ce294edeb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:17.725935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6c2fc568ee74d35bba01c6ce294edeb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:17.726623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6c2fc568ee74d35bba01c6ce294edeb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:17.739239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e6950562-bb91-483a-bea4-bfe1d1be4a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc30188074b04cda9f17003b47918994'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:17.739891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e6950562-bb91-483a-bea4-bfe1d1be4a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc30188074b04cda9f17003b47918994'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:17.753764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e6950562-bb91-483a-bea4-bfe1d1be4a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc30188074b04cda9f17003b47918994'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:17.759771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1211/2412] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:06:17 2022] DELETE /identity/v3/users/bc30188074b04cda9f17003b47918994 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:17.764502 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48307c-440e-46f6-acc4-06f2a44fbad7 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:17.764788 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48307c-440e-46f6-acc4-06f2a44fbad7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:17.765092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48307c-440e-46f6-acc4-06f2a44fbad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:17.784220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ec48307c-440e-46f6-acc4-06f2a44fbad7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:17.785458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ec48307c-440e-46f6-acc4-06f2a44fbad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:17.849349 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1212/2413] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:06:17.852701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6c2fc568ee74d35bba01c6ce294edeb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:17.859739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48307c-440e-46f6-acc4-06f2a44fbad7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:17.861202 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ec48307c-440e-46f6-acc4-06f2a44fbad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:17.866582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6c2fc568ee74d35bba01c6ce294edeb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:17.880056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1202/2414] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:06:17 2022] DELETE /identity/v3/users/d6c2fc568ee74d35bba01c6ce294edeb => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:17.922959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48307c-440e-46f6-acc4-06f2a44fbad7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:17.974901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a19120-0eda-412b-92f0-86cf921ae7c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:17.975512 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a19120-0eda-412b-92f0-86cf921ae7c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:17.975792 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a19120-0eda-412b-92f0-86cf921ae7c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:17.976064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a19120-0eda-412b-92f0-86cf921ae7c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.021395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:18.038444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a19120-0eda-412b-92f0-86cf921ae7c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:18.039981 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a19120-0eda-412b-92f0-86cf921ae7c4 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:18.050030 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1213/2415] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:06:18.145540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:18.146976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:18.161391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36af07d-34fd-47a7-b1ff-9487e845c3cb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:18.167210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:18.168270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.168550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.168830 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] PATH_INFO: `/v3/projects/90177d2267614a389ae39922cba114e2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.170075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=90177d2267614a389ae39922cba114e2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:18.171977 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:18.267731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90177d2267614a389ae39922cba114e2', 'request_id': 'req-e2008b79-f9e1-42cb-b850-11fb2a935afd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:18.274657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90177d2267614a389ae39922cba114e2', 'request_id': 'req-e2008b79-f9e1-42cb-b850-11fb2a935afd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:18.283564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90177d2267614a389ae39922cba114e2', 'request_id': 'req-e2008b79-f9e1-42cb-b850-11fb2a935afd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:18.350081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] The token cache is being invalidate because project 90177d2267614a389ae39922cba114e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:06:18.350369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:18.351011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e2008b79-f9e1-42cb-b850-11fb2a935afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:18.352530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1203/2416] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:06:18 2022] DELETE /identity/v3/projects/90177d2267614a389ae39922cba114e2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:18.358392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6615d0-26eb-4a49-ad87-f256f646c3b7 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.358392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6615d0-26eb-4a49-ad87-f256f646c3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.358943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6615d0-26eb-4a49-ad87-f256f646c3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.379038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1c6615d0-26eb-4a49-ad87-f256f646c3b7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:18.380260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1c6615d0-26eb-4a49-ad87-f256f646c3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:18.439711 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1214/2417] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:18.447652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950b17-5cd7-4374-a879-d27a5462fa00 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.447964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950b17-5cd7-4374-a879-d27a5462fa00 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.448352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950b17-5cd7-4374-a879-d27a5462fa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.468436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6615d0-26eb-4a49-ad87-f256f646c3b7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:18.470004 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1c6615d0-26eb-4a49-ad87-f256f646c3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:18.481450 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-71950b17-5cd7-4374-a879-d27a5462fa00 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:18.482842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-71950b17-5cd7-4374-a879-d27a5462fa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:18.487101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6615d0-26eb-4a49-ad87-f256f646c3b7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:18.491942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:18.493050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.493050 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.494180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] PATH_INFO: `/v3/projects/9784c47469514770908039754ceace65` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.494437 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9784c47469514770908039754ceace65)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:18.496863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:18.514088 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1204/2418] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:18.518142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71950b17-5cd7-4374-a879-d27a5462fa00 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:18.528503 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71950b17-5cd7-4374-a879-d27a5462fa00 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:18.533314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e20fe2-8698-48ca-a5e7-2c74fa3fe48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:18.533929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e20fe2-8698-48ca-a5e7-2c74fa3fe48b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.534213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e20fe2-8698-48ca-a5e7-2c74fa3fe48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.534485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e20fe2-8698-48ca-a5e7-2c74fa3fe48b admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.535130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e20fe2-8698-48ca-a5e7-2c74fa3fe48b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:06:18.535435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e20fe2-8698-48ca-a5e7-2c74fa3fe48b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:18.537036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e20fe2-8698-48ca-a5e7-2c74fa3fe48b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:18.549273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1205/2419] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:06:18.553480 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e20fe2-8698-48ca-a5e7-2c74fa3fe48b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:18.563924 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e20fe2-8698-48ca-a5e7-2c74fa3fe48b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:18.568643 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347ee0c-08bd-41e6-8676-a60f52203a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:18.569287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347ee0c-08bd-41e6-8676-a60f52203a9e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.569572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347ee0c-08bd-41e6-8676-a60f52203a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.569834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347ee0c-08bd-41e6-8676-a60f52203a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.570461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347ee0c-08bd-41e6-8676-a60f52203a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:18.572331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347ee0c-08bd-41e6-8676-a60f52203a9e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:18.586492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:18.588820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9784c47469514770908039754ceace65', 'request_id': 'req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:18.595978 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9784c47469514770908039754ceace65', 'request_id': 'req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:18.602641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9784c47469514770908039754ceace65', 'request_id': 'req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:18.672202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1206/2420] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:06:18 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:06:18.676298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347ee0c-08bd-41e6-8676-a60f52203a9e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:18.688284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] The token cache is being invalidate because project 9784c47469514770908039754ceace65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:06:18.688579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:18.692327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b96f990d-04d2-4c0c-b55f-b82ca07ad6aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:18.692327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1215/2421] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:06:18 2022] DELETE /identity/v3/projects/9784c47469514770908039754ceace65 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:06:18.697309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac883a6-8af7-4ec3-919d-6b0b9c9da1b0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.697733 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac883a6-8af7-4ec3-919d-6b0b9c9da1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.698085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac883a6-8af7-4ec3-919d-6b0b9c9da1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.718790 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-fac883a6-8af7-4ec3-919d-6b0b9c9da1b0 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:18.720012 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-fac883a6-8af7-4ec3-919d-6b0b9c9da1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:18.741787 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347ee0c-08bd-41e6-8676-a60f52203a9e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:18.747917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd9dee-5b67-492d-984b-82a500a92334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:18.748532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd9dee-5b67-492d-984b-82a500a92334 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.748816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd9dee-5b67-492d-984b-82a500a92334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.749167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd9dee-5b67-492d-984b-82a500a92334 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.750154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd9dee-5b67-492d-984b-82a500a92334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:18.752832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd9dee-5b67-492d-984b-82a500a92334 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:18.780529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1216/2422] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:06:18.789344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c7ed2-36d4-4038-9a5e-0131cb639abd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.789784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c7ed2-36d4-4038-9a5e-0131cb639abd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.790064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c7ed2-36d4-4038-9a5e-0131cb639abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.799222 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:18.810809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ed8c7ed2-36d4-4038-9a5e-0131cb639abd None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:18.812036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ed8c7ed2-36d4-4038-9a5e-0131cb639abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:18.819953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1217/2423] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:06:18.824635 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c7ed2-36d4-4038-9a5e-0131cb639abd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:18.833337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c7ed2-36d4-4038-9a5e-0131cb639abd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:18.836432 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1207/2424] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:06:18 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:06:18.838237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c3268-8295-4cec-a7c8-4d89642de1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:18.838835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c3268-8295-4cec-a7c8-4d89642de1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.839738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c3268-8295-4cec-a7c8-4d89642de1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.840023 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c3268-8295-4cec-a7c8-4d89642de1d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.840673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c3268-8295-4cec-a7c8-4d89642de1d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:06:18.840982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c3268-8295-4cec-a7c8-4d89642de1d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:18.841259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd9dee-5b67-492d-984b-82a500a92334 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:18.843077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a9bd9dee-5b67-492d-984b-82a500a92334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:18.843450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c3268-8295-4cec-a7c8-4d89642de1d4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:18.855811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd9dee-5b67-492d-984b-82a500a92334 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:18.860111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1218/2425] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:06:18.861068 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c3268-8295-4cec-a7c8-4d89642de1d4 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:18.861183 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9fcfc-0e60-45e8-a197-eb2708c7fffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:18.861692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9fcfc-0e60-45e8-a197-eb2708c7fffd admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.862089 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9fcfc-0e60-45e8-a197-eb2708c7fffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.862366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9fcfc-0e60-45e8-a197-eb2708c7fffd admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.863056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9fcfc-0e60-45e8-a197-eb2708c7fffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:18.864629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9fcfc-0e60-45e8-a197-eb2708c7fffd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:18.871481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c3268-8295-4cec-a7c8-4d89642de1d4 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:18.876438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c7d2a-7ab6-4f09-95a1-aa88dca95593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:18.877171 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c7d2a-7ab6-4f09-95a1-aa88dca95593 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.877456 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c7d2a-7ab6-4f09-95a1-aa88dca95593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.877732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c7d2a-7ab6-4f09-95a1-aa88dca95593 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.878360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c7d2a-7ab6-4f09-95a1-aa88dca95593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:18.878974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1208/2426] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:18.879876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9fcfc-0e60-45e8-a197-eb2708c7fffd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:18.880449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c7d2a-7ab6-4f09-95a1-aa88dca95593 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:18.889717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9fcfc-0e60-45e8-a197-eb2708c7fffd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:18.894408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848982d9-cfb4-4618-9c76-f2f2406d5dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:18.894996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848982d9-cfb4-4618-9c76-f2f2406d5dff admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.895277 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848982d9-cfb4-4618-9c76-f2f2406d5dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.895547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848982d9-cfb4-4618-9c76-f2f2406d5dff admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.896212 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848982d9-cfb4-4618-9c76-f2f2406d5dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:18.898033 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848982d9-cfb4-4618-9c76-f2f2406d5dff admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:18.908693 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1209/2427] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:18.912701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848982d9-cfb4-4618-9c76-f2f2406d5dff admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:18.922934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848982d9-cfb4-4618-9c76-f2f2406d5dff admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:18.927619 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29628f-7ff9-4d1b-9391-15973c402c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:18.928243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29628f-7ff9-4d1b-9391-15973c402c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:18.928522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29628f-7ff9-4d1b-9391-15973c402c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:18.928794 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29628f-7ff9-4d1b-9391-15973c402c31 admin admin] PATH_INFO: `/v3/projects/4e86de5c2b4949c4b791beb73c1d7c72/users/a4e6d682e3d34765847533d74248b8c0/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:18.948386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29628f-7ff9-4d1b-9391-15973c402c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e86de5c2b4949c4b791beb73c1d7c72, user_id=a4e6d682e3d34765847533d74248b8c0, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:18.948386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29628f-7ff9-4d1b-9391-15973c402c31 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:18.975240 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1219/2428] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:06:18 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:06:18.979103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c7d2a-7ab6-4f09-95a1-aa88dca95593 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.025166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1210/2429] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:18 2022] PUT /identity/v3/projects/4e86de5c2b4949c4b791beb73c1d7c72/users/a4e6d682e3d34765847533d74248b8c0/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:06:19.036566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9e976-dcd7-4481-915d-b7482ed2d403 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.036845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9e976-dcd7-4481-915d-b7482ed2d403 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.037147 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9e976-dcd7-4481-915d-b7482ed2d403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.042948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c7d2a-7ab6-4f09-95a1-aa88dca95593 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.065844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ead9e976-dcd7-4481-915d-b7482ed2d403 None None] MFA Rules not processed for user `a4e6d682e3d34765847533d74248b8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:19.067072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ead9e976-dcd7-4481-915d-b7482ed2d403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:19.099553 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a4733-d4d6-474b-8ba0-8c066a99bce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.100155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a4733-d4d6-474b-8ba0-8c066a99bce7 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.100429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a4733-d4d6-474b-8ba0-8c066a99bce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.100702 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a4733-d4d6-474b-8ba0-8c066a99bce7 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.101360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a4733-d4d6-474b-8ba0-8c066a99bce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.103303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a4733-d4d6-474b-8ba0-8c066a99bce7 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:19.131430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1211/2430] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:06:19.135805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9e976-dcd7-4481-915d-b7482ed2d403 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.146708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9e976-dcd7-4481-915d-b7482ed2d403 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.151225 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0442f1-20d9-4ee8-adbe-21af21f0e9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.151824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0442f1-20d9-4ee8-adbe-21af21f0e9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.152101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0442f1-20d9-4ee8-adbe-21af21f0e9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.152372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0442f1-20d9-4ee8-adbe-21af21f0e9f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.152988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0442f1-20d9-4ee8-adbe-21af21f0e9f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.154916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0442f1-20d9-4ee8-adbe-21af21f0e9f7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:19.207913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1220/2431] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:06:18 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:19.211942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a4733-d4d6-474b-8ba0-8c066a99bce7 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.221668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a4733-d4d6-474b-8ba0-8c066a99bce7 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.226300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca250388-1d67-4711-8cd8-88568005f676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.226876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca250388-1d67-4711-8cd8-88568005f676 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.227158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca250388-1d67-4711-8cd8-88568005f676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.227434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca250388-1d67-4711-8cd8-88568005f676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.228066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca250388-1d67-4711-8cd8-88568005f676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.229702 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca250388-1d67-4711-8cd8-88568005f676 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:19.239538 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1221/2432] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:06:19.243289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1212/2433] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:06:19 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:06:19.246503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca250388-1d67-4711-8cd8-88568005f676 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.247665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0442f1-20d9-4ee8-adbe-21af21f0e9f7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.252062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:19.254612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca250388-1d67-4711-8cd8-88568005f676 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.299754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:19.309531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0442f1-20d9-4ee8-adbe-21af21f0e9f7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.312241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7bf2f-5d61-43c6-88f8-075f44274098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.312959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7bf2f-5d61-43c6-88f8-075f44274098 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.313264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7bf2f-5d61-43c6-88f8-075f44274098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.313533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7bf2f-5d61-43c6-88f8-075f44274098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.314169 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7bf2f-5d61-43c6-88f8-075f44274098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.314285 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1f7d8-2388-49b6-b7a6-862e61341e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.314773 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1f7d8-2388-49b6-b7a6-862e61341e88 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.315055 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1f7d8-2388-49b6-b7a6-862e61341e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.315333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1f7d8-2388-49b6-b7a6-862e61341e88 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.315800 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a7bf2f-5d61-43c6-88f8-075f44274098 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:19.316010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1f7d8-2388-49b6-b7a6-862e61341e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.317927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1f7d8-2388-49b6-b7a6-862e61341e88 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:19.326319 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1222/2434] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:06:19.330509 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7bf2f-5d61-43c6-88f8-075f44274098 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.340460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a7bf2f-5d61-43c6-88f8-075f44274098 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.345447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37764552-9aa1-4bac-9505-8f47228eec53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.346076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37764552-9aa1-4bac-9505-8f47228eec53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.346362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37764552-9aa1-4bac-9505-8f47228eec53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.346637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37764552-9aa1-4bac-9505-8f47228eec53 admin admin] PATH_INFO: `/v3/projects/93eee20443c14b739e06abe3eba34c44/users/1f3839edcb09423b8b4885dffbd87af9/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.361460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37764552-9aa1-4bac-9505-8f47228eec53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93eee20443c14b739e06abe3eba34c44, user_id=1f3839edcb09423b8b4885dffbd87af9, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.364067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37764552-9aa1-4bac-9505-8f47228eec53 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:19.437065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1213/2435] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:06:19 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:06:19.440892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1f7d8-2388-49b6-b7a6-862e61341e88 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.447436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1223/2436] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:19 2022] PUT /identity/v3/projects/93eee20443c14b739e06abe3eba34c44/users/1f3839edcb09423b8b4885dffbd87af9/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:19.452993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767907f-9492-478b-b9fd-6ab272b6e8fb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.453217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1f7d8-2388-49b6-b7a6-862e61341e88 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.453409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767907f-9492-478b-b9fd-6ab272b6e8fb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.453693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767907f-9492-478b-b9fd-6ab272b6e8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.473310 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-5767907f-9492-478b-b9fd-6ab272b6e8fb None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:19.474734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5767907f-9492-478b-b9fd-6ab272b6e8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:19.511863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210d560-3cb3-4dc5-9a39-01fc95f152e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.512477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210d560-3cb3-4dc5-9a39-01fc95f152e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.512758 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210d560-3cb3-4dc5-9a39-01fc95f152e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.513065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210d560-3cb3-4dc5-9a39-01fc95f152e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.513742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210d560-3cb3-4dc5-9a39-01fc95f152e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.515335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210d560-3cb3-4dc5-9a39-01fc95f152e0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:19.525435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1214/2437] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:06:19.529449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210d560-3cb3-4dc5-9a39-01fc95f152e0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.537458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1224/2438] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:06:19.544834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210d560-3cb3-4dc5-9a39-01fc95f152e0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.546716 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bb95c-1704-4874-b27b-ca2e05ace51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.547533 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bb95c-1704-4874-b27b-ca2e05ace51a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.547816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bb95c-1704-4874-b27b-ca2e05ace51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.548123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bb95c-1704-4874-b27b-ca2e05ace51a admin admin] PATH_INFO: `/v3/projects/fb65817a93a345a9abef98c38a0eb405/users/11c987fdc3f84696aaa898fc1efc543d/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.553498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767907f-9492-478b-b9fd-6ab272b6e8fb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.554916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5767907f-9492-478b-b9fd-6ab272b6e8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:19.564492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bb95c-1704-4874-b27b-ca2e05ace51a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb65817a93a345a9abef98c38a0eb405, user_id=11c987fdc3f84696aaa898fc1efc543d, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.567144 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bb95c-1704-4874-b27b-ca2e05ace51a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:19.612391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767907f-9492-478b-b9fd-6ab272b6e8fb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.616220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb80ca0-6861-477e-9e3e-71838a77d235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.616846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb80ca0-6861-477e-9e3e-71838a77d235 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.617370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb80ca0-6861-477e-9e3e-71838a77d235 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.617645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb80ca0-6861-477e-9e3e-71838a77d235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.627443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb80ca0-6861-477e-9e3e-71838a77d235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.628864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb80ca0-6861-477e-9e3e-71838a77d235 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:19.638924 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1225/2439] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:06:19.647112 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1215/2440] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:19 2022] PUT /identity/v3/projects/fb65817a93a345a9abef98c38a0eb405/users/11c987fdc3f84696aaa898fc1efc543d/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:06:19.648379 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb80ca0-6861-477e-9e3e-71838a77d235 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.660066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb80ca0-6861-477e-9e3e-71838a77d235 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.721927 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4cc741-9a92-475f-b79a-25588cb56df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.722279 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cc741-9a92-475f-b79a-25588cb56df4 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.722558 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cc741-9a92-475f-b79a-25588cb56df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.722867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cc741-9a92-475f-b79a-25588cb56df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.723513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cc741-9a92-475f-b79a-25588cb56df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.725137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cc741-9a92-475f-b79a-25588cb56df4 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:19.731585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:19.737047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1226/2441] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:19.741545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bb95c-1704-4874-b27b-ca2e05ace51a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.752886 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bb95c-1704-4874-b27b-ca2e05ace51a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.758074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4247ef-0245-4048-918a-da0272a44992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.758847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4247ef-0245-4048-918a-da0272a44992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.759327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4247ef-0245-4048-918a-da0272a44992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.759796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4247ef-0245-4048-918a-da0272a44992 admin admin] PATH_INFO: `/v3/projects/fb65817a93a345a9abef98c38a0eb405/users/11c987fdc3f84696aaa898fc1efc543d/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.762485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4247ef-0245-4048-918a-da0272a44992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb65817a93a345a9abef98c38a0eb405, user_id=11c987fdc3f84696aaa898fc1efc543d, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.765397 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4247ef-0245-4048-918a-da0272a44992 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:19.776780 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1216/2442] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:19 2022] PUT /identity/v3/projects/fb65817a93a345a9abef98c38a0eb405/users/11c987fdc3f84696aaa898fc1efc543d/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:06:19.787834 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1799ad-1ce3-493a-b21f-b3d8ea199ac8 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.788123 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1799ad-1ce3-493a-b21f-b3d8ea199ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.788397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1799ad-1ce3-493a-b21f-b3d8ea199ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.817535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3c1799ad-1ce3-493a-b21f-b3d8ea199ac8 None None] MFA Rules not processed for user `11c987fdc3f84696aaa898fc1efc543d`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:19.818776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3c1799ad-1ce3-493a-b21f-b3d8ea199ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:19.904241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1227/2443] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:19.915553 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4247ef-0245-4048-918a-da0272a44992 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:19.918001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ed4247ef-0245-4048-918a-da0272a44992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:19.978373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4247ef-0245-4048-918a-da0272a44992 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:19.983593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199fbcb-02be-4776-ba1f-f2fb3e7b3248 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:19.990674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fbcb-02be-4776-ba1f-f2fb3e7b3248 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:19.990674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fbcb-02be-4776-ba1f-f2fb3e7b3248 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:19.990674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199fbcb-02be-4776-ba1f-f2fb3e7b3248 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:19.991701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:19.996704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199fbcb-02be-4776-ba1f-f2fb3e7b3248 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:19.998200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199fbcb-02be-4776-ba1f-f2fb3e7b3248 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:20.008735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1217/2444] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:20.990296 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1799ad-1ce3-493a-b21f-b3d8ea199ac8 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:21.079449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1799ad-1ce3-493a-b21f-b3d8ea199ac8 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:21.085530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:21.086185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:21.086472 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:21.086746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] PATH_INFO: `/v3/users/a4e6d682e3d34765847533d74248b8c0` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:21.087878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e6d682e3d34765847533d74248b8c0)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:21.090196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:21.275145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e6d682e3d34765847533d74248b8c0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:21.283479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e6d682e3d34765847533d74248b8c0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:21.344490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e6d682e3d34765847533d74248b8c0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:21.345239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e6d682e3d34765847533d74248b8c0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:21.366047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e6d682e3d34765847533d74248b8c0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:21.372682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1228/2445] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:06:20 2022] DELETE /identity/v3/users/a4e6d682e3d34765847533d74248b8c0 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:21.377270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198157ca-7937-458d-af67-4d3eb5f2a240 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:21.377770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198157ca-7937-458d-af67-4d3eb5f2a240 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:21.378032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198157ca-7937-458d-af67-4d3eb5f2a240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:21.399917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-198157ca-7937-458d-af67-4d3eb5f2a240 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:21.402146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-198157ca-7937-458d-af67-4d3eb5f2a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:21.471408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1218/2446] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:21.482517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:21.484031 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:21.496785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:21.546324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5192db-f576-4ec2-b421-1218d182db67 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:21.550176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0775374-a69d-4fe1-9e14-3dc7226a61a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:21.550788 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0775374-a69d-4fe1-9e14-3dc7226a61a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:21.551099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0775374-a69d-4fe1-9e14-3dc7226a61a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:21.551376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0775374-a69d-4fe1-9e14-3dc7226a61a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:21.561371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0775374-a69d-4fe1-9e14-3dc7226a61a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:21.562820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0775374-a69d-4fe1-9e14-3dc7226a61a8 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:21.572738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1229/2447] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:22.016697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198157ca-7937-458d-af67-4d3eb5f2a240 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:22.017993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-198157ca-7937-458d-af67-4d3eb5f2a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:22.030188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198157ca-7937-458d-af67-4d3eb5f2a240 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:22.036757 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:22.037929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:22.038357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:22.038774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] PATH_INFO: `/v3/projects/4e86de5c2b4949c4b791beb73c1d7c72` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:22.040232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e86de5c2b4949c4b791beb73c1d7c72)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:22.043249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:22.169192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e86de5c2b4949c4b791beb73c1d7c72', 'request_id': 'req-f810d539-d45d-4fef-b15c-20515c43db9c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:22.177876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e86de5c2b4949c4b791beb73c1d7c72', 'request_id': 'req-f810d539-d45d-4fef-b15c-20515c43db9c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:22.184732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e86de5c2b4949c4b791beb73c1d7c72', 'request_id': 'req-f810d539-d45d-4fef-b15c-20515c43db9c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:22.254317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] The token cache is being invalidate because project 4e86de5c2b4949c4b791beb73c1d7c72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:06:22.254673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:22.259134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:22.259134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1219/2448] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:06:22 2022] DELETE /identity/v3/projects/4e86de5c2b4949c4b791beb73c1d7c72 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:22.262673 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0775374-a69d-4fe1-9e14-3dc7226a61a8 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:22.272554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0775374-a69d-4fe1-9e14-3dc7226a61a8 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:22.331382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e87883-2042-43d8-bd8a-e0624f61090a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:22.332055 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e87883-2042-43d8-bd8a-e0624f61090a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:22.332298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e87883-2042-43d8-bd8a-e0624f61090a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:22.332571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e87883-2042-43d8-bd8a-e0624f61090a admin admin] PATH_INFO: `/v3/users/11c987fdc3f84696aaa898fc1efc543d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:22.333738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e87883-2042-43d8-bd8a-e0624f61090a admin admin] RBAC: Authorizing `identity:delete_user(user_id=11c987fdc3f84696aaa898fc1efc543d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:22.335665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e87883-2042-43d8-bd8a-e0624f61090a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:22.544880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-38e87883-2042-43d8-bd8a-e0624f61090a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11c987fdc3f84696aaa898fc1efc543d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:22.553326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-38e87883-2042-43d8-bd8a-e0624f61090a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11c987fdc3f84696aaa898fc1efc543d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:22.613800 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-38e87883-2042-43d8-bd8a-e0624f61090a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11c987fdc3f84696aaa898fc1efc543d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:22.614425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-38e87883-2042-43d8-bd8a-e0624f61090a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11c987fdc3f84696aaa898fc1efc543d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:22.633093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-38e87883-2042-43d8-bd8a-e0624f61090a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11c987fdc3f84696aaa898fc1efc543d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:22.640817 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1230/2449] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:06:22 2022] DELETE /identity/v3/users/11c987fdc3f84696aaa898fc1efc543d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:23.358565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:23.360915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:23.381530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810d539-d45d-4fef-b15c-20515c43db9c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:23.452228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:23.452997 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:23.453382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:23.454004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] PATH_INFO: `/v3/projects/fb65817a93a345a9abef98c38a0eb405` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:23.454922 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb65817a93a345a9abef98c38a0eb405)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:23.456841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:23.481177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:23.584045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb65817a93a345a9abef98c38a0eb405', 'request_id': 'req-aadda633-b4df-471a-9e5e-35b2a0e26b06'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:23.590344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb65817a93a345a9abef98c38a0eb405', 'request_id': 'req-aadda633-b4df-471a-9e5e-35b2a0e26b06'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:23.599519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb65817a93a345a9abef98c38a0eb405', 'request_id': 'req-aadda633-b4df-471a-9e5e-35b2a0e26b06'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:23.681707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] The token cache is being invalidate because project fb65817a93a345a9abef98c38a0eb405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:06:23.682162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:23.684332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aadda633-b4df-471a-9e5e-35b2a0e26b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:23.692378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad7d74-90b2-4c52-bc6b-070c53cebb69 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:23.692378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad7d74-90b2-4c52-bc6b-070c53cebb69 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:23.692378 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad7d74-90b2-4c52-bc6b-070c53cebb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:23.695388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1220/2450] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:06:23 2022] DELETE /identity/v3/projects/fb65817a93a345a9abef98c38a0eb405 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:23.715570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ffad7d74-90b2-4c52-bc6b-070c53cebb69 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:23.716809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ffad7d74-90b2-4c52-bc6b-070c53cebb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:23.739543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:23.790009 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1231/2451] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:06:23.797976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69592228-c72c-4518-8dba-ebebf19c7567 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:23.798327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69592228-c72c-4518-8dba-ebebf19c7567 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:23.798891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69592228-c72c-4518-8dba-ebebf19c7567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:23.833116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-69592228-c72c-4518-8dba-ebebf19c7567 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:23.834901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-69592228-c72c-4518-8dba-ebebf19c7567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:23.862546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1221/2452] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:23.865383 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad7d74-90b2-4c52-bc6b-070c53cebb69 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:23.879494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad7d74-90b2-4c52-bc6b-070c53cebb69 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:23.890831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b62a2-6599-493f-92b5-fd04c98a8c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:23.892821 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b62a2-6599-493f-92b5-fd04c98a8c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:23.893148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b62a2-6599-493f-92b5-fd04c98a8c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:23.893426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b62a2-6599-493f-92b5-fd04c98a8c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:23.894141 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b62a2-6599-493f-92b5-fd04c98a8c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:06:23.894443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b62a2-6599-493f-92b5-fd04c98a8c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:23.896107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b62a2-6599-493f-92b5-fd04c98a8c7d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:23.916025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1232/2453] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:06:23.919726 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69592228-c72c-4518-8dba-ebebf19c7567 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:23.932146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69592228-c72c-4518-8dba-ebebf19c7567 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:23.939869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b37f16-835a-4ed5-be1b-993472bea2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:23.940596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b37f16-835a-4ed5-be1b-993472bea2ea admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:23.940940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b37f16-835a-4ed5-be1b-993472bea2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:23.941299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b37f16-835a-4ed5-be1b-993472bea2ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:23.942046 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b37f16-835a-4ed5-be1b-993472bea2ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:23.944100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b37f16-835a-4ed5-be1b-993472bea2ea admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:24.071377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1222/2454] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:06:23 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:24.075829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b62a2-6599-493f-92b5-fd04c98a8c7d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:24.095874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b62a2-6599-493f-92b5-fd04c98a8c7d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:24.199751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d1a42-914f-4a07-8a7f-229d3e59081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:24.200425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d1a42-914f-4a07-8a7f-229d3e59081f admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:24.200703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d1a42-914f-4a07-8a7f-229d3e59081f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:24.201030 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d1a42-914f-4a07-8a7f-229d3e59081f admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:24.201701 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d1a42-914f-4a07-8a7f-229d3e59081f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:24.204042 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d1a42-914f-4a07-8a7f-229d3e59081f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:24.277861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1233/2455] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:06:24 2022] POST /identity/v3/users => generated 283 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:24.282662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b37f16-835a-4ed5-be1b-993472bea2ea admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:24.295404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b37f16-835a-4ed5-be1b-993472bea2ea admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:24.302991 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cef50-2eee-4d58-a450-8233221bc9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:24.303797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cef50-2eee-4d58-a450-8233221bc9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:24.304082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cef50-2eee-4d58-a450-8233221bc9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:24.304359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cef50-2eee-4d58-a450-8233221bc9f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:24.305034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cef50-2eee-4d58-a450-8233221bc9f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:24.306664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cef50-2eee-4d58-a450-8233221bc9f7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:24.321143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1223/2456] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:24.322465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d1a42-914f-4a07-8a7f-229d3e59081f admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:24.342217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d1a42-914f-4a07-8a7f-229d3e59081f admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:24.347974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f219ca-9852-4bfd-bf2a-7020031a9fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:24.348588 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f219ca-9852-4bfd-bf2a-7020031a9fee admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:24.348866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f219ca-9852-4bfd-bf2a-7020031a9fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:24.349174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f219ca-9852-4bfd-bf2a-7020031a9fee admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:24.349882 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f219ca-9852-4bfd-bf2a-7020031a9fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:24.351453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f219ca-9852-4bfd-bf2a-7020031a9fee admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:24.363490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1234/2457] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:24.367508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cef50-2eee-4d58-a450-8233221bc9f7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:24.379006 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cef50-2eee-4d58-a450-8233221bc9f7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:24.383954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa5d89-79f8-45e8-9c20-df068610b41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:24.384745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa5d89-79f8-45e8-9c20-df068610b41d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:24.385167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa5d89-79f8-45e8-9c20-df068610b41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:24.385506 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa5d89-79f8-45e8-9c20-df068610b41d admin admin] PATH_INFO: `/v3/projects/0ece850165f148e988e9b86a143a9c4c/users/90f35819735d4458ac3b6d0cd5a946c9/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:24.400747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa5d89-79f8-45e8-9c20-df068610b41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ece850165f148e988e9b86a143a9c4c, user_id=90f35819735d4458ac3b6d0cd5a946c9, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:24.404189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa5d89-79f8-45e8-9c20-df068610b41d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:24.493502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1224/2458] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:24 2022] PUT /identity/v3/projects/0ece850165f148e988e9b86a143a9c4c/users/90f35819735d4458ac3b6d0cd5a946c9/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:24.505748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417903c-254a-4a12-b0ba-e3306ffd429f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:24.506036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417903c-254a-4a12-b0ba-e3306ffd429f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:24.506369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417903c-254a-4a12-b0ba-e3306ffd429f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:24.538888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e417903c-254a-4a12-b0ba-e3306ffd429f None None] MFA Rules not processed for user `90f35819735d4458ac3b6d0cd5a946c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:24.540218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e417903c-254a-4a12-b0ba-e3306ffd429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:24.638527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1235/2459] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:24.643540 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa5d89-79f8-45e8-9c20-df068610b41d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:24.719521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417903c-254a-4a12-b0ba-e3306ffd429f None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:24.720962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e417903c-254a-4a12-b0ba-e3306ffd429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:24.744080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa5d89-79f8-45e8-9c20-df068610b41d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:24.750697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ebd31-a0e6-4614-9bf1-38626c4c7cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:24.752859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ebd31-a0e6-4614-9bf1-38626c4c7cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:24.752859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ebd31-a0e6-4614-9bf1-38626c4c7cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:24.753357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ebd31-a0e6-4614-9bf1-38626c4c7cf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:24.754041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ebd31-a0e6-4614-9bf1-38626c4c7cf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:24.754851 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:24.757347 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ebd31-a0e6-4614-9bf1-38626c4c7cf2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:24.806831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417903c-254a-4a12-b0ba-e3306ffd429f None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:24.812046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b765b5-03e0-4845-8735-5981fe7aac3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:24.812681 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b765b5-03e0-4845-8735-5981fe7aac3f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:24.812962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b765b5-03e0-4845-8735-5981fe7aac3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:24.813286 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b765b5-03e0-4845-8735-5981fe7aac3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:24.815568 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-97b765b5-03e0-4845-8735-5981fe7aac3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:24.869729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1225/2460] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:06:24 2022] POST /identity/v3/projects => generated 363 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:24.872794 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ebd31-a0e6-4614-9bf1-38626c4c7cf2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:24.888294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b765b5-03e0-4845-8735-5981fe7aac3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:24.889985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b765b5-03e0-4845-8735-5981fe7aac3f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:24.903641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1236/2461] 10.222.0.17 () {60 vars in 1396 bytes} [Thu Aug 11 10:06:24 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:06:24.915061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:24.953469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ebd31-a0e6-4614-9bf1-38626c4c7cf2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:24.958462 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb56dee-169c-4bb8-a71a-4fd66d98a9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:24.959084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb56dee-169c-4bb8-a71a-4fd66d98a9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:24.959597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb56dee-169c-4bb8-a71a-4fd66d98a9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:24.959772 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb56dee-169c-4bb8-a71a-4fd66d98a9f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:24.960420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb56dee-169c-4bb8-a71a-4fd66d98a9f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:24.962380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb56dee-169c-4bb8-a71a-4fd66d98a9f4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:25.068993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1226/2462] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:06:24 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:25.073152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b765b5-03e0-4845-8735-5981fe7aac3f service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:25.084101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b765b5-03e0-4845-8735-5981fe7aac3f service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:25.101144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82909482-f71e-4d8f-bb26-8d1d55a09fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:25.101815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82909482-f71e-4d8f-bb26-8d1d55a09fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:25.102098 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82909482-f71e-4d8f-bb26-8d1d55a09fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:25.102371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82909482-f71e-4d8f-bb26-8d1d55a09fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:25.103032 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82909482-f71e-4d8f-bb26-8d1d55a09fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:25.104934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82909482-f71e-4d8f-bb26-8d1d55a09fb6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:25.117062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1237/2463] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:25.122292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb56dee-169c-4bb8-a71a-4fd66d98a9f4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:25.134958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb56dee-169c-4bb8-a71a-4fd66d98a9f4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:25.141300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d210ca-83f4-4129-9433-1b6333508282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:25.142425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d210ca-83f4-4129-9433-1b6333508282 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:25.142425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d210ca-83f4-4129-9433-1b6333508282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:25.142764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d210ca-83f4-4129-9433-1b6333508282 admin admin] PATH_INFO: `/v3/projects/b1ffb01ec26a428bbba0f70e3034dc5b/users/10cab503826d48ef8fbe1b6fa8562d40/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:25.179245 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d210ca-83f4-4129-9433-1b6333508282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ffb01ec26a428bbba0f70e3034dc5b, user_id=10cab503826d48ef8fbe1b6fa8562d40, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:25.182282 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d210ca-83f4-4129-9433-1b6333508282 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:25.266339 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1227/2464] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:25 2022] PUT /identity/v3/projects/b1ffb01ec26a428bbba0f70e3034dc5b/users/10cab503826d48ef8fbe1b6fa8562d40/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:25.270425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82909482-f71e-4d8f-bb26-8d1d55a09fb6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:25.281200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82909482-f71e-4d8f-bb26-8d1d55a09fb6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:25.350306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd246c-5ebd-481f-b700-c8f25e350285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:25.351012 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd246c-5ebd-481f-b700-c8f25e350285 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:25.351575 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd246c-5ebd-481f-b700-c8f25e350285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:25.351697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd246c-5ebd-481f-b700-c8f25e350285 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:25.352406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd246c-5ebd-481f-b700-c8f25e350285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:25.354274 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd246c-5ebd-481f-b700-c8f25e350285 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:25.371467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d210ca-83f4-4129-9433-1b6333508282 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:25.375734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1238/2465] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:25.384147 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d210ca-83f4-4129-9433-1b6333508282 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:25.389390 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05894d23-73ef-48b1-8e73-0bfa551f3660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:25.390409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05894d23-73ef-48b1-8e73-0bfa551f3660 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:25.390925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05894d23-73ef-48b1-8e73-0bfa551f3660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:25.391408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05894d23-73ef-48b1-8e73-0bfa551f3660 admin admin] PATH_INFO: `/v3/projects/b1ffb01ec26a428bbba0f70e3034dc5b/users/10cab503826d48ef8fbe1b6fa8562d40/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:25.393670 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05894d23-73ef-48b1-8e73-0bfa551f3660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ffb01ec26a428bbba0f70e3034dc5b, user_id=10cab503826d48ef8fbe1b6fa8562d40, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:25.396656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05894d23-73ef-48b1-8e73-0bfa551f3660 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:25.418597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1228/2466] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:25 2022] PUT /identity/v3/projects/b1ffb01ec26a428bbba0f70e3034dc5b/users/10cab503826d48ef8fbe1b6fa8562d40/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:25.420100 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f915d5f-3619-421b-934a-3a13bc783132 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:25.420392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f915d5f-3619-421b-934a-3a13bc783132 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:25.420666 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f915d5f-3619-421b-934a-3a13bc783132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:25.453930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3f915d5f-3619-421b-934a-3a13bc783132 None None] MFA Rules not processed for user `10cab503826d48ef8fbe1b6fa8562d40`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:25.455188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3f915d5f-3619-421b-934a-3a13bc783132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:25.516205 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:25.545272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1239/2467] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:06:25.556645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05894d23-73ef-48b1-8e73-0bfa551f3660 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:25.558669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-05894d23-73ef-48b1-8e73-0bfa551f3660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:25.639159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05894d23-73ef-48b1-8e73-0bfa551f3660 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:25.644266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f140d9-ae50-4d60-9461-b4b74fc5f745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:25.644915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f140d9-ae50-4d60-9461-b4b74fc5f745 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:25.645234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f140d9-ae50-4d60-9461-b4b74fc5f745 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:25.645514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f140d9-ae50-4d60-9461-b4b74fc5f745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:25.666625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f140d9-ae50-4d60-9461-b4b74fc5f745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:25.670135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f140d9-ae50-4d60-9461-b4b74fc5f745 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:25.694274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1229/2468] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:25.782038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fffe71-0662-46c5-b3d3-c6cde140647c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:25.782459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fffe71-0662-46c5-b3d3-c6cde140647c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:25.782839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fffe71-0662-46c5-b3d3-c6cde140647c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:25.820356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-22fffe71-0662-46c5-b3d3-c6cde140647c None None] MFA Rules not processed for user `1f3839edcb09423b8b4885dffbd87af9`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:25.821906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-22fffe71-0662-46c5-b3d3-c6cde140647c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:25.900477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1240/2469] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:25.905451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f140d9-ae50-4d60-9461-b4b74fc5f745 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:25.906881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d2f140d9-ae50-4d60-9461-b4b74fc5f745 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:25.931220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fffe71-0662-46c5-b3d3-c6cde140647c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:25.950014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fffe71-0662-46c5-b3d3-c6cde140647c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:25.954725 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2260ed3-efbc-46d6-8c69-fc943a23f051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:25.955440 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2260ed3-efbc-46d6-8c69-fc943a23f051 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:25.955723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2260ed3-efbc-46d6-8c69-fc943a23f051 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:25.955992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2260ed3-efbc-46d6-8c69-fc943a23f051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:25.986358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f140d9-ae50-4d60-9461-b4b74fc5f745 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:25.993457 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca9d05-5c1e-440e-8c13-b53fed0ff8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:25.994155 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9d05-5c1e-440e-8c13-b53fed0ff8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:25.994459 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9d05-5c1e-440e-8c13-b53fed0ff8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:25.994732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9d05-5c1e-440e-8c13-b53fed0ff8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:25.995917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca9d05-5c1e-440e-8c13-b53fed0ff8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:25.998022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca9d05-5c1e-440e-8c13-b53fed0ff8b0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:26.025402 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2260ed3-efbc-46d6-8c69-fc943a23f051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:26.026917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2260ed3-efbc-46d6-8c69-fc943a23f051 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:26.036967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1241/2470] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:06:26.063056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1230/2471] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:06:25 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:26.067001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2260ed3-efbc-46d6-8c69-fc943a23f051 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:26.079377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2260ed3-efbc-46d6-8c69-fc943a23f051 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:26.137406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce476b-d74d-48ef-8250-af732ce8a807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:26.138010 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce476b-d74d-48ef-8250-af732ce8a807 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:26.138293 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce476b-d74d-48ef-8250-af732ce8a807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:26.138566 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce476b-d74d-48ef-8250-af732ce8a807 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:26.139200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce476b-d74d-48ef-8250-af732ce8a807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:26.141130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce476b-d74d-48ef-8250-af732ce8a807 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:26.248143 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1242/2472] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:06:26 2022] POST /identity/v3/users => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:26.252161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca9d05-5c1e-440e-8c13-b53fed0ff8b0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:26.264790 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca9d05-5c1e-440e-8c13-b53fed0ff8b0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:26.270889 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c92947-3654-4e79-bd8d-d9089a393aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:26.271533 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c92947-3654-4e79-bd8d-d9089a393aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:26.275488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c92947-3654-4e79-bd8d-d9089a393aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:26.275488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c92947-3654-4e79-bd8d-d9089a393aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:26.275488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c92947-3654-4e79-bd8d-d9089a393aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:26.275488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c92947-3654-4e79-bd8d-d9089a393aa5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:26.278002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:26.291233 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1231/2473] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:06:26.292064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce476b-d74d-48ef-8250-af732ce8a807 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:26.299139 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:26.303747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce476b-d74d-48ef-8250-af732ce8a807 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:26.308172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669c4ff-0f5d-44b0-b15c-acf3f5f53ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:26.308933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669c4ff-0f5d-44b0-b15c-acf3f5f53ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:26.309258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669c4ff-0f5d-44b0-b15c-acf3f5f53ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:26.309535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669c4ff-0f5d-44b0-b15c-acf3f5f53ef2 admin admin] PATH_INFO: `/v3/projects/01ce5ccbff294fb1bef29e3d2785f254/users/dcf3c18f07704fbab2a34576e7d4d958/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:26.329466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669c4ff-0f5d-44b0-b15c-acf3f5f53ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ce5ccbff294fb1bef29e3d2785f254, user_id=dcf3c18f07704fbab2a34576e7d4d958, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:26.334954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669c4ff-0f5d-44b0-b15c-acf3f5f53ef2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:26.403036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1243/2474] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:26 2022] PUT /identity/v3/projects/01ce5ccbff294fb1bef29e3d2785f254/users/dcf3c18f07704fbab2a34576e7d4d958/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:06:26.407014 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c92947-3654-4e79-bd8d-d9089a393aa5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:26.420417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c92947-3654-4e79-bd8d-d9089a393aa5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:26.487682 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82acfc-0c63-4ffc-a24d-3c27fe23964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:26.488304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82acfc-0c63-4ffc-a24d-3c27fe23964d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:26.488585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82acfc-0c63-4ffc-a24d-3c27fe23964d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:26.488858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82acfc-0c63-4ffc-a24d-3c27fe23964d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:26.489625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82acfc-0c63-4ffc-a24d-3c27fe23964d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:26.491187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82acfc-0c63-4ffc-a24d-3c27fe23964d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:26.501190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1232/2475] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:26.507111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669c4ff-0f5d-44b0-b15c-acf3f5f53ef2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:26.518362 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669c4ff-0f5d-44b0-b15c-acf3f5f53ef2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:26.523126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f79092-740e-4a2a-8ff3-8db46488fb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:26.523807 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f79092-740e-4a2a-8ff3-8db46488fb61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:26.524085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f79092-740e-4a2a-8ff3-8db46488fb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:26.524361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f79092-740e-4a2a-8ff3-8db46488fb61 admin admin] PATH_INFO: `/v3/projects/01ce5ccbff294fb1bef29e3d2785f254/users/dcf3c18f07704fbab2a34576e7d4d958/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:26.526972 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f79092-740e-4a2a-8ff3-8db46488fb61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ce5ccbff294fb1bef29e3d2785f254, user_id=dcf3c18f07704fbab2a34576e7d4d958, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:26.529955 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f79092-740e-4a2a-8ff3-8db46488fb61 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:26.541165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1244/2476] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:26 2022] PUT /identity/v3/projects/01ce5ccbff294fb1bef29e3d2785f254/users/dcf3c18f07704fbab2a34576e7d4d958/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:06:32.584590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6317bb-8ad0-49a4-a408-eaa9e6d02e54 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:32.585255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6317bb-8ad0-49a4-a408-eaa9e6d02e54 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:32.585255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6317bb-8ad0-49a4-a408-eaa9e6d02e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:32.614747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ac6317bb-8ad0-49a4-a408-eaa9e6d02e54 None None] MFA Rules not processed for user `dcf3c18f07704fbab2a34576e7d4d958`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:32.616170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ac6317bb-8ad0-49a4-a408-eaa9e6d02e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:32.687944 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1233/2477] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:32.700488 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f79092-740e-4a2a-8ff3-8db46488fb61 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:32.702206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-58f79092-740e-4a2a-8ff3-8db46488fb61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:32.759612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f79092-740e-4a2a-8ff3-8db46488fb61 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:32.763432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d12aa-81f6-4387-ad33-2f7249456f9c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83448e4c6704f7c8771132bd67eaa1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:32.764033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d12aa-81f6-4387-ad33-2f7249456f9c service glance] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:32.764311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d12aa-81f6-4387-ad33-2f7249456f9c service glance] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:32.764584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d12aa-81f6-4387-ad33-2f7249456f9c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:32.821773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d12aa-81f6-4387-ad33-2f7249456f9c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:32.823442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d12aa-81f6-4387-ad33-2f7249456f9c service glance] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:32.833438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1245/2478] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:33.084642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2adfa-4c3c-4b70-b22e-4a518c4b62a6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:33.084932 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2adfa-4c3c-4b70-b22e-4a518c4b62a6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:33.085255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2adfa-4c3c-4b70-b22e-4a518c4b62a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:33.109424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-87c2adfa-4c3c-4b70-b22e-4a518c4b62a6 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:33.110642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-87c2adfa-4c3c-4b70-b22e-4a518c4b62a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:33.169820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1234/2479] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 10:06:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:33.287366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d12aa-81f6-4387-ad33-2f7249456f9c service glance] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:33.288771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f63d12aa-81f6-4387-ad33-2f7249456f9c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:33.360313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d12aa-81f6-4387-ad33-2f7249456f9c service glance] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:33.365327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53e52d-f897-4276-8478-f5098abb6122 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:33.365939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53e52d-f897-4276-8478-f5098abb6122 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:33.366220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53e52d-f897-4276-8478-f5098abb6122 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:33.366490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53e52d-f897-4276-8478-f5098abb6122 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:33.375662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53e52d-f897-4276-8478-f5098abb6122 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:33.377295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53e52d-f897-4276-8478-f5098abb6122 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:33.385367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1246/2480] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:06:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:06:37.334538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818382b0-c254-480d-bb1c-1f545518d278 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:37.334538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818382b0-c254-480d-bb1c-1f545518d278 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:37.335443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818382b0-c254-480d-bb1c-1f545518d278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:37.356359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-818382b0-c254-480d-bb1c-1f545518d278 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:37.357656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-818382b0-c254-480d-bb1c-1f545518d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:37.365988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1235/2481] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 10:06:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:37.383466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53e52d-f897-4276-8478-f5098abb6122 service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:37.395868 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53e52d-f897-4276-8478-f5098abb6122 service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:37.401838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59be915-f53c-48d6-84bb-119ab05f6536 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:37.402775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59be915-f53c-48d6-84bb-119ab05f6536 service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:37.403211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59be915-f53c-48d6-84bb-119ab05f6536 service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:37.403647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59be915-f53c-48d6-84bb-119ab05f6536 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:37.412598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59be915-f53c-48d6-84bb-119ab05f6536 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:37.414380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59be915-f53c-48d6-84bb-119ab05f6536 service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:37.418893 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:37.424756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1247/2482] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:06:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:06:40.260660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818382b0-c254-480d-bb1c-1f545518d278 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:40.311443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:40.384603 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818382b0-c254-480d-bb1c-1f545518d278 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:40.390360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:40.391468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:40.392499 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:40.392812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] PATH_INFO: `/v3/users/90f35819735d4458ac3b6d0cd5a946c9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:40.394268 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90f35819735d4458ac3b6d0cd5a946c9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:40.398434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:40.791668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f35819735d4458ac3b6d0cd5a946c9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:40.792430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f35819735d4458ac3b6d0cd5a946c9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:40.793154 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f35819735d4458ac3b6d0cd5a946c9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:40.898790 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f35819735d4458ac3b6d0cd5a946c9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:40.913942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f35819735d4458ac3b6d0cd5a946c9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:40.927312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1236/2483] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:06:40 2022] DELETE /identity/v3/users/90f35819735d4458ac3b6d0cd5a946c9 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:40.931993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d246c05-b47b-479b-8de6-61e7d7774f7d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:40.933121 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d246c05-b47b-479b-8de6-61e7d7774f7d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:40.933247 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d246c05-b47b-479b-8de6-61e7d7774f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:40.955262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9d246c05-b47b-479b-8de6-61e7d7774f7d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:40.956518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9d246c05-b47b-479b-8de6-61e7d7774f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:41.024947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1248/2484] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:41.036241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:41.037934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:41.099122 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba5039-0cc8-4add-8801-55261d32ead3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:41.102853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d8e03-8e1b-4a6e-b902-df3a2afbbce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:41.103514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8e03-8e1b-4a6e-b902-df3a2afbbce6 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:41.103859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8e03-8e1b-4a6e-b902-df3a2afbbce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:41.104764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8e03-8e1b-4a6e-b902-df3a2afbbce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:41.115365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d8e03-8e1b-4a6e-b902-df3a2afbbce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:41.116873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d8e03-8e1b-4a6e-b902-df3a2afbbce6 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:41.128425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1237/2485] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:41.942355 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d246c05-b47b-479b-8de6-61e7d7774f7d None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:41.944019 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9d246c05-b47b-479b-8de6-61e7d7774f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:41.963175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d246c05-b47b-479b-8de6-61e7d7774f7d None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:41.976838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:41.977819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:41.978200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:41.978650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] PATH_INFO: `/v3/projects/0ece850165f148e988e9b86a143a9c4c` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:41.982640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ece850165f148e988e9b86a143a9c4c)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:41.990175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:42.149656 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ece850165f148e988e9b86a143a9c4c', 'request_id': 'req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:42.164802 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ece850165f148e988e9b86a143a9c4c', 'request_id': 'req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:42.176097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ece850165f148e988e9b86a143a9c4c', 'request_id': 'req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:42.346029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] The token cache is being invalidate because project 0ece850165f148e988e9b86a143a9c4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:06:42.346442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:42.347109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:42.348624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1249/2486] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:06:41 2022] DELETE /identity/v3/projects/0ece850165f148e988e9b86a143a9c4c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:42.352733 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d8e03-8e1b-4a6e-b902-df3a2afbbce6 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:42.371197 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d8e03-8e1b-4a6e-b902-df3a2afbbce6 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:42.454265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:42.478452 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfc89c-8e05-423c-b6bd-e459f4e6c82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:42.480409 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfc89c-8e05-423c-b6bd-e459f4e6c82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:42.483654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfc89c-8e05-423c-b6bd-e459f4e6c82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:42.484120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfc89c-8e05-423c-b6bd-e459f4e6c82f admin admin] PATH_INFO: `/v3/users/10cab503826d48ef8fbe1b6fa8562d40` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:42.485597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfc89c-8e05-423c-b6bd-e459f4e6c82f admin admin] RBAC: Authorizing `identity:delete_user(user_id=10cab503826d48ef8fbe1b6fa8562d40)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:42.487783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfc89c-8e05-423c-b6bd-e459f4e6c82f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:42.716383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6ccfc89c-8e05-423c-b6bd-e459f4e6c82f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10cab503826d48ef8fbe1b6fa8562d40'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:42.718065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6ccfc89c-8e05-423c-b6bd-e459f4e6c82f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10cab503826d48ef8fbe1b6fa8562d40'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:42.719357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6ccfc89c-8e05-423c-b6bd-e459f4e6c82f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10cab503826d48ef8fbe1b6fa8562d40'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:42.813093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6ccfc89c-8e05-423c-b6bd-e459f4e6c82f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10cab503826d48ef8fbe1b6fa8562d40'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:42.833886 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6ccfc89c-8e05-423c-b6bd-e459f4e6c82f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10cab503826d48ef8fbe1b6fa8562d40'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:42.864053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1238/2487] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:06:42 2022] DELETE /identity/v3/users/10cab503826d48ef8fbe1b6fa8562d40 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:43.509683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:43.511500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:43.530032 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ae65c-f302-41b9-bcc3-54d74e0ba91e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:43.607165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:43.608082 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:43.608560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:43.609054 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] PATH_INFO: `/v3/projects/b1ffb01ec26a428bbba0f70e3034dc5b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:43.613308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1ffb01ec26a428bbba0f70e3034dc5b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:43.615406 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:43.732217 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:43.876953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1ffb01ec26a428bbba0f70e3034dc5b', 'request_id': 'req-b3f36e3c-a859-402c-b9cf-b859000b10e3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:43.889105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1ffb01ec26a428bbba0f70e3034dc5b', 'request_id': 'req-b3f36e3c-a859-402c-b9cf-b859000b10e3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:43.906999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1ffb01ec26a428bbba0f70e3034dc5b', 'request_id': 'req-b3f36e3c-a859-402c-b9cf-b859000b10e3'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:43.999941 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] The token cache is being invalidate because project b1ffb01ec26a428bbba0f70e3034dc5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:06:44.000399 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:44.006089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b3f36e3c-a859-402c-b9cf-b859000b10e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:06:44.009273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1250/2488] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:06:43 2022] DELETE /identity/v3/projects/b1ffb01ec26a428bbba0f70e3034dc5b => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:44.010689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15550308-867c-4284-b1ed-9dcb55dc5672 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:44.011165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15550308-867c-4284-b1ed-9dcb55dc5672 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:44.011385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15550308-867c-4284-b1ed-9dcb55dc5672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:44.068662 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-15550308-867c-4284-b1ed-9dcb55dc5672 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:44.079152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-15550308-867c-4284-b1ed-9dcb55dc5672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:44.184568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1239/2489] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:44.200291 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259a482-086d-4b15-bfe9-c336a56dd744 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:44.200731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259a482-086d-4b15-bfe9-c336a56dd744 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:44.201914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259a482-086d-4b15-bfe9-c336a56dd744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:44.230495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-8259a482-086d-4b15-bfe9-c336a56dd744 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:44.231958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8259a482-086d-4b15-bfe9-c336a56dd744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:44.240634 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1251/2490] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:44.245470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15550308-867c-4284-b1ed-9dcb55dc5672 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:44.262515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15550308-867c-4284-b1ed-9dcb55dc5672 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:44.281466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c402e-7bb7-47dd-b3f6-5920463022a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:44.282257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c402e-7bb7-47dd-b3f6-5920463022a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:44.282647 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c402e-7bb7-47dd-b3f6-5920463022a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:44.283039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c402e-7bb7-47dd-b3f6-5920463022a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:44.283783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c402e-7bb7-47dd-b3f6-5920463022a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:06:44.284207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c402e-7bb7-47dd-b3f6-5920463022a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:44.287870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c402e-7bb7-47dd-b3f6-5920463022a9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:44.317969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1240/2491] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:06:44.319389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259a482-086d-4b15-bfe9-c336a56dd744 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:44.333845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259a482-086d-4b15-bfe9-c336a56dd744 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:44.348239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9d016-6866-4022-9a2b-7d7de479143c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:44.349030 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9d016-6866-4022-9a2b-7d7de479143c admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:44.349415 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9d016-6866-4022-9a2b-7d7de479143c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:44.349783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9d016-6866-4022-9a2b-7d7de479143c admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:44.350514 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9d016-6866-4022-9a2b-7d7de479143c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:44.357445 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9d016-6866-4022-9a2b-7d7de479143c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:44.480108 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1252/2492] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:06:44 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:44.485431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c402e-7bb7-47dd-b3f6-5920463022a9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:44.504402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c402e-7bb7-47dd-b3f6-5920463022a9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:44.614991 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:44.644925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e49de-c098-48bb-88db-4fa96961110b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:44.645582 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e49de-c098-48bb-88db-4fa96961110b admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:44.645864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e49de-c098-48bb-88db-4fa96961110b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:44.646169 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e49de-c098-48bb-88db-4fa96961110b admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:44.646813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e49de-c098-48bb-88db-4fa96961110b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:44.648794 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e49de-c098-48bb-88db-4fa96961110b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:44.801139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1241/2493] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:06:44 2022] POST /identity/v3/users => generated 289 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:44.804948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9d016-6866-4022-9a2b-7d7de479143c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:44.821601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9d016-6866-4022-9a2b-7d7de479143c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:44.834152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0adc9b-a593-47d7-a40a-44ca4622500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:44.835998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0adc9b-a593-47d7-a40a-44ca4622500d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:44.835998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0adc9b-a593-47d7-a40a-44ca4622500d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:44.835998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0adc9b-a593-47d7-a40a-44ca4622500d admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:44.836331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0adc9b-a593-47d7-a40a-44ca4622500d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:44.838034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0adc9b-a593-47d7-a40a-44ca4622500d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:44.887232 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1253/2494] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:44.888856 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e49de-c098-48bb-88db-4fa96961110b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:44.907309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e49de-c098-48bb-88db-4fa96961110b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:44.912424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829d6a8-d05c-416f-8d64-fc6723b16d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:44.913226 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829d6a8-d05c-416f-8d64-fc6723b16d8c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:44.913625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829d6a8-d05c-416f-8d64-fc6723b16d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:44.914052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829d6a8-d05c-416f-8d64-fc6723b16d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:44.915078 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829d6a8-d05c-416f-8d64-fc6723b16d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:44.918337 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829d6a8-d05c-416f-8d64-fc6723b16d8c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:44.935434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1242/2495] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:44.936047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0adc9b-a593-47d7-a40a-44ca4622500d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:44.946876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0adc9b-a593-47d7-a40a-44ca4622500d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:44.952118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88310d5-0b58-4fbf-9b2e-5212880bcc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:44.952920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88310d5-0b58-4fbf-9b2e-5212880bcc6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:44.953302 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88310d5-0b58-4fbf-9b2e-5212880bcc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:44.953712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88310d5-0b58-4fbf-9b2e-5212880bcc6d admin admin] PATH_INFO: `/v3/projects/e19c257e01a04a4da046089ed09e61b7/users/a23a6f0be60647799828cd38bde5ebc9/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:44.971683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88310d5-0b58-4fbf-9b2e-5212880bcc6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19c257e01a04a4da046089ed09e61b7, user_id=a23a6f0be60647799828cd38bde5ebc9, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:44.974397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88310d5-0b58-4fbf-9b2e-5212880bcc6d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:45.087152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1254/2496] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:44 2022] PUT /identity/v3/projects/e19c257e01a04a4da046089ed09e61b7/users/a23a6f0be60647799828cd38bde5ebc9/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:45.090299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3724826-85fe-4a55-8dd8-04854b0862cb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:45.091010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3724826-85fe-4a55-8dd8-04854b0862cb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:45.091213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3724826-85fe-4a55-8dd8-04854b0862cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:45.127173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d3724826-85fe-4a55-8dd8-04854b0862cb None None] MFA Rules not processed for user `a23a6f0be60647799828cd38bde5ebc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:45.128718 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d3724826-85fe-4a55-8dd8-04854b0862cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:45.318810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1243/2497] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:45.326632 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88310d5-0b58-4fbf-9b2e-5212880bcc6d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:45.398594 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88310d5-0b58-4fbf-9b2e-5212880bcc6d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:45.403356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985c5e9-9754-4898-b24e-3b3e97c30ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:45.403968 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985c5e9-9754-4898-b24e-3b3e97c30ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:45.404248 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985c5e9-9754-4898-b24e-3b3e97c30ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:45.404551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985c5e9-9754-4898-b24e-3b3e97c30ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:45.405236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985c5e9-9754-4898-b24e-3b3e97c30ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:45.407144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985c5e9-9754-4898-b24e-3b3e97c30ec8 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:45.726914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1255/2498] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:06:45 2022] POST /identity/v3/projects => generated 377 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:45.728021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3724826-85fe-4a55-8dd8-04854b0862cb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:45.740150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3724826-85fe-4a55-8dd8-04854b0862cb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:45.802515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10288f3-d070-4b27-89c7-3ce202ac7490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:45.803364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10288f3-d070-4b27-89c7-3ce202ac7490 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:45.803644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10288f3-d070-4b27-89c7-3ce202ac7490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:45.803917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10288f3-d070-4b27-89c7-3ce202ac7490 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:45.804545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10288f3-d070-4b27-89c7-3ce202ac7490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:45.806635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10288f3-d070-4b27-89c7-3ce202ac7490 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:45.953989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1244/2499] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:06:45 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:06:45.955403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985c5e9-9754-4898-b24e-3b3e97c30ec8 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:45.967361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985c5e9-9754-4898-b24e-3b3e97c30ec8 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:45.972359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54914e2e-5af1-414b-9542-62fb6f5515e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:45.972995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54914e2e-5af1-414b-9542-62fb6f5515e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:45.973295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54914e2e-5af1-414b-9542-62fb6f5515e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:45.973569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54914e2e-5af1-414b-9542-62fb6f5515e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:45.974214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54914e2e-5af1-414b-9542-62fb6f5515e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:45.975803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54914e2e-5af1-414b-9542-62fb6f5515e0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:45.987184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1256/2500] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:45.991606 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10288f3-d070-4b27-89c7-3ce202ac7490 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:46.002600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10288f3-d070-4b27-89c7-3ce202ac7490 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:46.007306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a2ed-f3bc-4094-8155-801e05cb3fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:46.012570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a2ed-f3bc-4094-8155-801e05cb3fed admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:46.012570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a2ed-f3bc-4094-8155-801e05cb3fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:46.012570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a2ed-f3bc-4094-8155-801e05cb3fed admin admin] PATH_INFO: `/v3/projects/a18932efd5724e1098f75daf3802833c/users/b7d7994856c54d989ccc8510a73c2477/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:46.025285 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124a2ed-f3bc-4094-8155-801e05cb3fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18932efd5724e1098f75daf3802833c, user_id=b7d7994856c54d989ccc8510a73c2477, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:46.028700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124a2ed-f3bc-4094-8155-801e05cb3fed admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:46.118292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1245/2501] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:45 2022] PUT /identity/v3/projects/a18932efd5724e1098f75daf3802833c/users/b7d7994856c54d989ccc8510a73c2477/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:06:46.123008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54914e2e-5af1-414b-9542-62fb6f5515e0 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:46.134064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54914e2e-5af1-414b-9542-62fb6f5515e0 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:46.162251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:46.280135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2daabf-3009-40d4-bfa9-5ba7ba26f991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:46.280135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2daabf-3009-40d4-bfa9-5ba7ba26f991 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:46.280543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2daabf-3009-40d4-bfa9-5ba7ba26f991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:46.280918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2daabf-3009-40d4-bfa9-5ba7ba26f991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:46.282729 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2daabf-3009-40d4-bfa9-5ba7ba26f991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:46.288474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2daabf-3009-40d4-bfa9-5ba7ba26f991 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:46.311706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1257/2502] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:06:46.316041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a2ed-f3bc-4094-8155-801e05cb3fed admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:46.331311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a2ed-f3bc-4094-8155-801e05cb3fed admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:46.341699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6dc68-9596-4595-ab65-02368f6233f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:46.343750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6dc68-9596-4595-ab65-02368f6233f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:46.344311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6dc68-9596-4595-ab65-02368f6233f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:46.344657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6dc68-9596-4595-ab65-02368f6233f8 admin admin] PATH_INFO: `/v3/projects/a18932efd5724e1098f75daf3802833c/users/b7d7994856c54d989ccc8510a73c2477/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:46.348091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6dc68-9596-4595-ab65-02368f6233f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18932efd5724e1098f75daf3802833c, user_id=b7d7994856c54d989ccc8510a73c2477, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:46.356310 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6dc68-9596-4595-ab65-02368f6233f8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:46.397800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1246/2503] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:06:46 2022] PUT /identity/v3/projects/a18932efd5724e1098f75daf3802833c/users/b7d7994856c54d989ccc8510a73c2477/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:06:46.410470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852a421-4ce5-4c3e-a379-e5fed2bf6eb9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:46.410905 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852a421-4ce5-4c3e-a379-e5fed2bf6eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:46.411213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852a421-4ce5-4c3e-a379-e5fed2bf6eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:46.475870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3852a421-4ce5-4c3e-a379-e5fed2bf6eb9 None None] MFA Rules not processed for user `b7d7994856c54d989ccc8510a73c2477`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:46.486850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3852a421-4ce5-4c3e-a379-e5fed2bf6eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:46.675018 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1258/2504] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:46.685724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6dc68-9596-4595-ab65-02368f6233f8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:46.687901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b8d6dc68-9596-4595-ab65-02368f6233f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:46.758541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:46.848024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6dc68-9596-4595-ab65-02368f6233f8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:46.851917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e463f-4cd4-43b1-aa8a-7ab5ee8cf634 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:46.852528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e463f-4cd4-43b1-aa8a-7ab5ee8cf634 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:46.852806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e463f-4cd4-43b1-aa8a-7ab5ee8cf634 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:46.853126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e463f-4cd4-43b1-aa8a-7ab5ee8cf634 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:46.922527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e463f-4cd4-43b1-aa8a-7ab5ee8cf634 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:46.926671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e463f-4cd4-43b1-aa8a-7ab5ee8cf634 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:46.940391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1247/2505] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:47.374175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006e604-a5e0-4dc9-af00-a5a09ec0e84b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:47.374686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006e604-a5e0-4dc9-af00-a5a09ec0e84b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:47.374686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006e604-a5e0-4dc9-af00-a5a09ec0e84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:47.399968 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e006e604-a5e0-4dc9-af00-a5a09ec0e84b None None] MFA Rules not processed for user `1f3839edcb09423b8b4885dffbd87af9`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:47.405326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e006e604-a5e0-4dc9-af00-a5a09ec0e84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:47.475982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1259/2506] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:06:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:06:53.959889 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e463f-4cd4-43b1-aa8a-7ab5ee8cf634 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:53.962297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-578e463f-4cd4-43b1-aa8a-7ab5ee8cf634 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:53.976017 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e463f-4cd4-43b1-aa8a-7ab5ee8cf634 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:53.979975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fafbcc-7574-4ffd-9c02-00a03b2bce29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:53.980590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fafbcc-7574-4ffd-9c02-00a03b2bce29 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:53.980867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fafbcc-7574-4ffd-9c02-00a03b2bce29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:53.981503 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fafbcc-7574-4ffd-9c02-00a03b2bce29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:53.991365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fafbcc-7574-4ffd-9c02-00a03b2bce29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:53.992780 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fafbcc-7574-4ffd-9c02-00a03b2bce29 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:54.002779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1248/2507] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:06:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:06:57.465762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=78 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:06:57.734600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48333-2d32-492e-88af-448405477095 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:57.735294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48333-2d32-492e-88af-448405477095 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:57.735742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48333-2d32-492e-88af-448405477095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:57.760075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e7b48333-2d32-492e-88af-448405477095 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:06:57.761330 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e7b48333-2d32-492e-88af-448405477095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:57.830534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1260/2508] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 10:06:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:06:57.845873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fafbcc-7574-4ffd-9c02-00a03b2bce29 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:06:57.849026 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-59fafbcc-7574-4ffd-9c02-00a03b2bce29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:06:57.955804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fafbcc-7574-4ffd-9c02-00a03b2bce29 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:06:57.955804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546d116-c22d-4ea5-8e9b-89d623ade908 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:06:57.955804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546d116-c22d-4ea5-8e9b-89d623ade908 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:06:57.955804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546d116-c22d-4ea5-8e9b-89d623ade908 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:06:57.956321 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546d116-c22d-4ea5-8e9b-89d623ade908 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:06:57.962827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546d116-c22d-4ea5-8e9b-89d623ade908 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:06:57.964433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546d116-c22d-4ea5-8e9b-89d623ade908 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:06:57.973955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1249/2509] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:06:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 11 10:07:01.847114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48333-2d32-492e-88af-448405477095 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:01.849163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e7b48333-2d32-492e-88af-448405477095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:01.937099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48333-2d32-492e-88af-448405477095 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:01.941159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc93c4a-f12b-456d-b26c-2e0709d55a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:01.941781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc93c4a-f12b-456d-b26c-2e0709d55a0c service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:01.942049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc93c4a-f12b-456d-b26c-2e0709d55a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:01.942417 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc93c4a-f12b-456d-b26c-2e0709d55a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:01.945517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dbc93c4a-f12b-456d-b26c-2e0709d55a0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:02.067954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc93c4a-f12b-456d-b26c-2e0709d55a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:02.069889 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc93c4a-f12b-456d-b26c-2e0709d55a0c service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:02.082006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1261/2510] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:07.383041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546d116-c22d-4ea5-8e9b-89d623ade908 service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:07.384436 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d546d116-c22d-4ea5-8e9b-89d623ade908 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:07.400343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546d116-c22d-4ea5-8e9b-89d623ade908 service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:07.406956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3c848-73fe-4893-b492-fcbc2a4a45e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:07.408111 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3c848-73fe-4893-b492-fcbc2a4a45e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:07.408601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3c848-73fe-4893-b492-fcbc2a4a45e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:07.409123 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3c848-73fe-4893-b492-fcbc2a4a45e9 admin admin] PATH_INFO: `/v3/users/28b31650305943b498f02c882baa0144` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:07.410490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3c848-73fe-4893-b492-fcbc2a4a45e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b31650305943b498f02c882baa0144)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:07.412801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3c848-73fe-4893-b492-fcbc2a4a45e9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:07.643240 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:07.655015 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-12f3c848-73fe-4893-b492-fcbc2a4a45e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b31650305943b498f02c882baa0144'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:07.655675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-12f3c848-73fe-4893-b492-fcbc2a4a45e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b31650305943b498f02c882baa0144'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:07.656289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-12f3c848-73fe-4893-b492-fcbc2a4a45e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b31650305943b498f02c882baa0144'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:07.720325 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-12f3c848-73fe-4893-b492-fcbc2a4a45e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b31650305943b498f02c882baa0144'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:07.733514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-12f3c848-73fe-4893-b492-fcbc2a4a45e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b31650305943b498f02c882baa0144'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:07.745628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1250/2511] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:07 2022] DELETE /identity/v3/users/28b31650305943b498f02c882baa0144 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:08.031561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc93c4a-f12b-456d-b26c-2e0709d55a0c service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:08.045002 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc93c4a-f12b-456d-b26c-2e0709d55a0c service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:08.094561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:08.101180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:08.101806 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:08.102084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:08.102354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] PATH_INFO: `/v3/projects/c22d997be9cb4bbb8235eda18fa92074` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:08.103478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c22d997be9cb4bbb8235eda18fa92074)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:08.105391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:08.221794 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c22d997be9cb4bbb8235eda18fa92074', 'request_id': 'req-cca53d7e-cf4b-4f3a-86d5-f0022942d275'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:08.228133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c22d997be9cb4bbb8235eda18fa92074', 'request_id': 'req-cca53d7e-cf4b-4f3a-86d5-f0022942d275'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:08.235600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c22d997be9cb4bbb8235eda18fa92074', 'request_id': 'req-cca53d7e-cf4b-4f3a-86d5-f0022942d275'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:08.332892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] The token cache is being invalidate because project c22d997be9cb4bbb8235eda18fa92074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:08.333152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:08.333773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-cca53d7e-cf4b-4f3a-86d5-f0022942d275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:08.335343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1262/2512] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:08 2022] DELETE /identity/v3/projects/c22d997be9cb4bbb8235eda18fa92074 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:08.340338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0ed01-cfbd-48c2-9cce-7ed554376b59 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:08.340649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0ed01-cfbd-48c2-9cce-7ed554376b59 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:08.340924 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0ed01-cfbd-48c2-9cce-7ed554376b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:08.361528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c3e0ed01-cfbd-48c2-9cce-7ed554376b59 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:08.362742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c3e0ed01-cfbd-48c2-9cce-7ed554376b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:08.425491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1251/2513] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:08.433265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65a130-bc91-45da-b78b-5f6f05521f12 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:08.433561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65a130-bc91-45da-b78b-5f6f05521f12 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:08.433822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65a130-bc91-45da-b78b-5f6f05521f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:08.455070 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-dc65a130-bc91-45da-b78b-5f6f05521f12 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:08.456300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dc65a130-bc91-45da-b78b-5f6f05521f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:08.464430 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1263/2514] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:08.469683 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0ed01-cfbd-48c2-9cce-7ed554376b59 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:08.479668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0ed01-cfbd-48c2-9cce-7ed554376b59 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:08.485303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b930bf-a0fc-4c83-b3db-1b687624acb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:08.485952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b930bf-a0fc-4c83-b3db-1b687624acb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:08.486236 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b930bf-a0fc-4c83-b3db-1b687624acb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:08.486514 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b930bf-a0fc-4c83-b3db-1b687624acb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:08.487173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b930bf-a0fc-4c83-b3db-1b687624acb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:07:08.487478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b930bf-a0fc-4c83-b3db-1b687624acb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:08.489256 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b930bf-a0fc-4c83-b3db-1b687624acb3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:08.502144 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1252/2515] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:07:08.506529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65a130-bc91-45da-b78b-5f6f05521f12 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:08.516097 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65a130-bc91-45da-b78b-5f6f05521f12 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:08.520810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9baa0a-3bd3-4d73-8fbc-8db9c646c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:08.527130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9baa0a-3bd3-4d73-8fbc-8db9c646c6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:08.527130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9baa0a-3bd3-4d73-8fbc-8db9c646c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:08.527130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9baa0a-3bd3-4d73-8fbc-8db9c646c6f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:08.527130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9baa0a-3bd3-4d73-8fbc-8db9c646c6f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:08.527130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9baa0a-3bd3-4d73-8fbc-8db9c646c6f1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:08.603498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1264/2516] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:08 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:08.608595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b930bf-a0fc-4c83-b3db-1b687624acb3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:08.619008 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b930bf-a0fc-4c83-b3db-1b687624acb3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:08.680713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f14f7-ea2d-4f71-b33a-30063fe9f1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:08.681373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f14f7-ea2d-4f71-b33a-30063fe9f1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:08.681826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f14f7-ea2d-4f71-b33a-30063fe9f1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:08.682100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f14f7-ea2d-4f71-b33a-30063fe9f1e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:08.682723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f14f7-ea2d-4f71-b33a-30063fe9f1e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:08.684650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f14f7-ea2d-4f71-b33a-30063fe9f1e4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:08.778301 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1253/2517] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:08 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:08.782840 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9baa0a-3bd3-4d73-8fbc-8db9c646c6f1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:08.792952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9baa0a-3bd3-4d73-8fbc-8db9c646c6f1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:08.798155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294496e-b813-4902-bc62-af1ccd366bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:08.798742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294496e-b813-4902-bc62-af1ccd366bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:08.799021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294496e-b813-4902-bc62-af1ccd366bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:08.799287 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294496e-b813-4902-bc62-af1ccd366bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:08.799925 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294496e-b813-4902-bc62-af1ccd366bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:08.801540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294496e-b813-4902-bc62-af1ccd366bd2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:08.811234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1265/2518] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:08.815068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f14f7-ea2d-4f71-b33a-30063fe9f1e4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:08.824983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f14f7-ea2d-4f71-b33a-30063fe9f1e4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:08.829926 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf6e0b-a900-4be4-bd69-6ee8cb1125ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:08.830526 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf6e0b-a900-4be4-bd69-6ee8cb1125ae admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:08.830807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf6e0b-a900-4be4-bd69-6ee8cb1125ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:08.831082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf6e0b-a900-4be4-bd69-6ee8cb1125ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:08.831727 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf6e0b-a900-4be4-bd69-6ee8cb1125ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:08.833362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf6e0b-a900-4be4-bd69-6ee8cb1125ae admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:08.844777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1254/2519] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:08.848880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294496e-b813-4902-bc62-af1ccd366bd2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:08.858697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294496e-b813-4902-bc62-af1ccd366bd2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:08.863263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff752b24-cbbd-46df-91cd-e267bd14fd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:08.863897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff752b24-cbbd-46df-91cd-e267bd14fd41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:08.864176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff752b24-cbbd-46df-91cd-e267bd14fd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:08.864445 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff752b24-cbbd-46df-91cd-e267bd14fd41 admin admin] PATH_INFO: `/v3/projects/0617d55240c24660b0475b82899106a5/users/f84d7dbc7980400596447791314463d0/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:08.879203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff752b24-cbbd-46df-91cd-e267bd14fd41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0617d55240c24660b0475b82899106a5, user_id=f84d7dbc7980400596447791314463d0, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:08.881856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff752b24-cbbd-46df-91cd-e267bd14fd41 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:08.966874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1266/2520] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:08 2022] PUT /identity/v3/projects/0617d55240c24660b0475b82899106a5/users/f84d7dbc7980400596447791314463d0/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:08.977309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a3f2-51ac-4d4c-81ee-5c47f1f320db None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:08.977600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a3f2-51ac-4d4c-81ee-5c47f1f320db None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:08.977900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a3f2-51ac-4d4c-81ee-5c47f1f320db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:09.006702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e2b6a3f2-51ac-4d4c-81ee-5c47f1f320db None None] MFA Rules not processed for user `f84d7dbc7980400596447791314463d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:09.007918 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e2b6a3f2-51ac-4d4c-81ee-5c47f1f320db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:09.011862 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:09.072887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1255/2521] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:09.077555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff752b24-cbbd-46df-91cd-e267bd14fd41 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:09.140543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff752b24-cbbd-46df-91cd-e267bd14fd41 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:09.145244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553c293-a4a6-4e37-8d55-68e415af39f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:09.145846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553c293-a4a6-4e37-8d55-68e415af39f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:09.146126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553c293-a4a6-4e37-8d55-68e415af39f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:09.146398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553c293-a4a6-4e37-8d55-68e415af39f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:09.147033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553c293-a4a6-4e37-8d55-68e415af39f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:09.148995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3553c293-a4a6-4e37-8d55-68e415af39f1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:09.242699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1267/2522] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:09 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:09.246909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6a3f2-51ac-4d4c-81ee-5c47f1f320db None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:09.258581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6a3f2-51ac-4d4c-81ee-5c47f1f320db None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:09.316388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e25db6-e697-4d42-b24e-0263e3781311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:09.317021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e25db6-e697-4d42-b24e-0263e3781311 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:09.317312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e25db6-e697-4d42-b24e-0263e3781311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:09.317586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e25db6-e697-4d42-b24e-0263e3781311 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:09.318206 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e25db6-e697-4d42-b24e-0263e3781311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:09.320110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e25db6-e697-4d42-b24e-0263e3781311 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:09.390006 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1256/2523] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:09 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:09.394106 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553c293-a4a6-4e37-8d55-68e415af39f1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:09.404496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553c293-a4a6-4e37-8d55-68e415af39f1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:09.409081 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e743b53-3a70-4698-9e57-ff0305fb6d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:09.409674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e743b53-3a70-4698-9e57-ff0305fb6d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:09.409956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e743b53-3a70-4698-9e57-ff0305fb6d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:09.410225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e743b53-3a70-4698-9e57-ff0305fb6d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:09.410859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e743b53-3a70-4698-9e57-ff0305fb6d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:09.412446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e743b53-3a70-4698-9e57-ff0305fb6d9c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:09.413709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:09.423295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1268/2524] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:09.436032 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e25db6-e697-4d42-b24e-0263e3781311 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:09.446308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e25db6-e697-4d42-b24e-0263e3781311 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:09.451580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde98e-8d28-4268-b95e-e2f0aec07fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:09.456443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde98e-8d28-4268-b95e-e2f0aec07fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:09.456443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde98e-8d28-4268-b95e-e2f0aec07fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:09.456443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffde98e-8d28-4268-b95e-e2f0aec07fc4 admin admin] PATH_INFO: `/v3/projects/aa7861a957ac442caab4249622dbb87f/users/cc6640b6b57f49fdadd43ce683e9e1c5/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:09.467574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffde98e-8d28-4268-b95e-e2f0aec07fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7861a957ac442caab4249622dbb87f, user_id=cc6640b6b57f49fdadd43ce683e9e1c5, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:09.470627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffde98e-8d28-4268-b95e-e2f0aec07fc4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:09.538901 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1257/2525] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:09 2022] PUT /identity/v3/projects/aa7861a957ac442caab4249622dbb87f/users/cc6640b6b57f49fdadd43ce683e9e1c5/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:09.543194 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e743b53-3a70-4698-9e57-ff0305fb6d9c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:09.553857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e743b53-3a70-4698-9e57-ff0305fb6d9c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:09.612116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79733c82-4096-4803-9745-2a61afaf7c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:09.612717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79733c82-4096-4803-9745-2a61afaf7c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:09.612996 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79733c82-4096-4803-9745-2a61afaf7c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:09.613301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79733c82-4096-4803-9745-2a61afaf7c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:09.613940 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79733c82-4096-4803-9745-2a61afaf7c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:09.615513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79733c82-4096-4803-9745-2a61afaf7c4f admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:09.626718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1269/2526] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:09.630992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde98e-8d28-4268-b95e-e2f0aec07fc4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:09.643497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffde98e-8d28-4268-b95e-e2f0aec07fc4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:09.651694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbe6ad-5324-409a-8edc-a86659445a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:09.651694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbe6ad-5324-409a-8edc-a86659445a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:09.651694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbe6ad-5324-409a-8edc-a86659445a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:09.651694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbe6ad-5324-409a-8edc-a86659445a72 admin admin] PATH_INFO: `/v3/projects/aa7861a957ac442caab4249622dbb87f/users/cc6640b6b57f49fdadd43ce683e9e1c5/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:09.652102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbe6ad-5324-409a-8edc-a86659445a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa7861a957ac442caab4249622dbb87f, user_id=cc6640b6b57f49fdadd43ce683e9e1c5, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:09.655688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabbe6ad-5324-409a-8edc-a86659445a72 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:09.666732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1258/2527] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:09 2022] PUT /identity/v3/projects/aa7861a957ac442caab4249622dbb87f/users/cc6640b6b57f49fdadd43ce683e9e1c5/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:09.679005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4345a-6138-49da-a22a-d34c60089842 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:09.679299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4345a-6138-49da-a22a-d34c60089842 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:09.679572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4345a-6138-49da-a22a-d34c60089842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:09.708375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4ff4345a-6138-49da-a22a-d34c60089842 None None] MFA Rules not processed for user `cc6640b6b57f49fdadd43ce683e9e1c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:09.709646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4ff4345a-6138-49da-a22a-d34c60089842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:09.784867 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1270/2528] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:09.797319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbe6ad-5324-409a-8edc-a86659445a72 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:09.799100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-dabbe6ad-5324-409a-8edc-a86659445a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:09.863655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbe6ad-5324-409a-8edc-a86659445a72 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:09.867351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980a719-6d6c-484f-b39a-456ee8081786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:09.867976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980a719-6d6c-484f-b39a-456ee8081786 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:09.868264 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980a719-6d6c-484f-b39a-456ee8081786 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:09.868525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980a719-6d6c-484f-b39a-456ee8081786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:09.879508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980a719-6d6c-484f-b39a-456ee8081786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:09.880940 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980a719-6d6c-484f-b39a-456ee8081786 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:09.886520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:09.892101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1259/2529] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:10.846768 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4345a-6138-49da-a22a-d34c60089842 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:10.910364 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4345a-6138-49da-a22a-d34c60089842 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:10.915305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:10.916133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:10.916533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:10.916811 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] PATH_INFO: `/v3/users/f84d7dbc7980400596447791314463d0` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:10.918077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84d7dbc7980400596447791314463d0)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:10.920026 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:10.944656 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:11.089660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84d7dbc7980400596447791314463d0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:11.096764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84d7dbc7980400596447791314463d0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:11.161154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84d7dbc7980400596447791314463d0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:11.161812 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84d7dbc7980400596447791314463d0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:11.174869 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84d7dbc7980400596447791314463d0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:11.183637 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1271/2530] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:10 2022] DELETE /identity/v3/users/f84d7dbc7980400596447791314463d0 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:11.186092 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c96c1-e958-44ea-a694-973a30d383bd None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:11.186563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c96c1-e958-44ea-a694-973a30d383bd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:11.186669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c96c1-e958-44ea-a694-973a30d383bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:11.206149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0a2c96c1-e958-44ea-a694-973a30d383bd None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:11.207391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0a2c96c1-e958-44ea-a694-973a30d383bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:11.272843 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1260/2531] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:11.282627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:11.290316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:11.340865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb86095-0476-4bb8-8298-a0024154af58 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:11.344882 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7de20-a92b-401d-becc-47bab0f40065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:11.345560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7de20-a92b-401d-becc-47bab0f40065 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:11.345839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7de20-a92b-401d-becc-47bab0f40065 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:11.346117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7de20-a92b-401d-becc-47bab0f40065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:11.355859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7de20-a92b-401d-becc-47bab0f40065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:11.357775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7de20-a92b-401d-becc-47bab0f40065 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:11.368782 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1272/2532] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:11.830953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c96c1-e958-44ea-a694-973a30d383bd None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:11.832566 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0a2c96c1-e958-44ea-a694-973a30d383bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:11.846053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c96c1-e958-44ea-a694-973a30d383bd None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:11.851270 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:11.851916 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:11.852195 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:11.852624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] PATH_INFO: `/v3/projects/0617d55240c24660b0475b82899106a5` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:11.853806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0617d55240c24660b0475b82899106a5)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:11.855939 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:12.002093 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0617d55240c24660b0475b82899106a5', 'request_id': 'req-307c0efe-dee3-47e3-8d13-c8783eb5e492'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:12.008108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0617d55240c24660b0475b82899106a5', 'request_id': 'req-307c0efe-dee3-47e3-8d13-c8783eb5e492'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:12.015263 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0617d55240c24660b0475b82899106a5', 'request_id': 'req-307c0efe-dee3-47e3-8d13-c8783eb5e492'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:12.093623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] The token cache is being invalidate because project 0617d55240c24660b0475b82899106a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:12.093921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:12.094585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-307c0efe-dee3-47e3-8d13-c8783eb5e492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:12.096345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1261/2533] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:11 2022] DELETE /identity/v3/projects/0617d55240c24660b0475b82899106a5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:12.100238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7de20-a92b-401d-becc-47bab0f40065 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:12.112183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7de20-a92b-401d-becc-47bab0f40065 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:12.172771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:12.173382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:12.173721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:12.174047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] PATH_INFO: `/v3/users/cc6640b6b57f49fdadd43ce683e9e1c5` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:12.175199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc6640b6b57f49fdadd43ce683e9e1c5)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:12.177142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:12.319971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:12.325465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc6640b6b57f49fdadd43ce683e9e1c5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:12.333189 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc6640b6b57f49fdadd43ce683e9e1c5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:12.395665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc6640b6b57f49fdadd43ce683e9e1c5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:12.396300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc6640b6b57f49fdadd43ce683e9e1c5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:12.410895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc6640b6b57f49fdadd43ce683e9e1c5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:12.417196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1273/2534] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:12 2022] DELETE /identity/v3/users/cc6640b6b57f49fdadd43ce683e9e1c5 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:12.851899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6720260-b197-4222-85ab-5f844e292e6f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:12.856116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6720260-b197-4222-85ab-5f844e292e6f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:12.856116 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6720260-b197-4222-85ab-5f844e292e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:12.856462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:12.858621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:12.883181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a6720260-b197-4222-85ab-5f844e292e6f None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:12.884487 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a6720260-b197-4222-85ab-5f844e292e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:12.940769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1481152-9356-4eba-9099-5fe66efb4c2d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:12.944695 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1262/2535] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 10:07:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:12.945497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:12.946158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:12.946428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:12.946701 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] PATH_INFO: `/v3/projects/aa7861a957ac442caab4249622dbb87f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:12.947835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa7861a957ac442caab4249622dbb87f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:12.949796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:12.957622 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6720260-b197-4222-85ab-5f844e292e6f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:12.959075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a6720260-b197-4222-85ab-5f844e292e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:12.996578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:13.037593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6720260-b197-4222-85ab-5f844e292e6f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:13.042953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3a7bc-9fd3-428b-bf98-4726e3a97ac2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:13.043571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3a7bc-9fd3-428b-bf98-4726e3a97ac2 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:13.043854 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3a7bc-9fd3-428b-bf98-4726e3a97ac2 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:13.044138 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3a7bc-9fd3-428b-bf98-4726e3a97ac2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:13.052822 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3a7bc-9fd3-428b-bf98-4726e3a97ac2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:13.054437 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3a7bc-9fd3-428b-bf98-4726e3a97ac2 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:13.063232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1263/2536] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:07:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 11 10:07:13.078275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa7861a957ac442caab4249622dbb87f', 'request_id': 'req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:13.083846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa7861a957ac442caab4249622dbb87f', 'request_id': 'req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:13.091339 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa7861a957ac442caab4249622dbb87f', 'request_id': 'req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:13.219250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] The token cache is being invalidate because project aa7861a957ac442caab4249622dbb87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:13.219250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:13.219250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0f7ad086-0d2b-4e4f-afbb-d954138ad88a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:13.219250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1274/2537] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:12 2022] DELETE /identity/v3/projects/aa7861a957ac442caab4249622dbb87f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:13.224357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ceb01-065b-4510-8798-076dfc9ed41f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:13.224842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ceb01-065b-4510-8798-076dfc9ed41f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:13.225324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ceb01-065b-4510-8798-076dfc9ed41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:13.258764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fd8ceb01-065b-4510-8798-076dfc9ed41f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:13.260015 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fd8ceb01-065b-4510-8798-076dfc9ed41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:13.387109 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1264/2538] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:13.437648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4c7ba-bb30-4927-8fb5-873dd738ad47 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:13.437648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4c7ba-bb30-4927-8fb5-873dd738ad47 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:13.437648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4c7ba-bb30-4927-8fb5-873dd738ad47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:13.437648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-71b4c7ba-bb30-4927-8fb5-873dd738ad47 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:13.437648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-71b4c7ba-bb30-4927-8fb5-873dd738ad47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:13.446455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1275/2539] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:13.476031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ceb01-065b-4510-8798-076dfc9ed41f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:13.493728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ceb01-065b-4510-8798-076dfc9ed41f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:13.501786 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5abfc-48fb-4b5f-bdfe-93751002339b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:13.502416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5abfc-48fb-4b5f-bdfe-93751002339b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:13.502744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5abfc-48fb-4b5f-bdfe-93751002339b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:13.502972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5abfc-48fb-4b5f-bdfe-93751002339b admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:13.503635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5abfc-48fb-4b5f-bdfe-93751002339b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:07:13.512132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5abfc-48fb-4b5f-bdfe-93751002339b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:13.514243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5abfc-48fb-4b5f-bdfe-93751002339b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:13.547560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4c7ba-bb30-4927-8fb5-873dd738ad47 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:13.548026 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1265/2540] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:07:13.573107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4c7ba-bb30-4927-8fb5-873dd738ad47 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:13.573107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb987cfa-efb1-43ee-9171-26bf8ff0dffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:13.573107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb987cfa-efb1-43ee-9171-26bf8ff0dffc admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:13.573107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb987cfa-efb1-43ee-9171-26bf8ff0dffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:13.573508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb987cfa-efb1-43ee-9171-26bf8ff0dffc admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:13.573508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb987cfa-efb1-43ee-9171-26bf8ff0dffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:13.573508 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb987cfa-efb1-43ee-9171-26bf8ff0dffc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:13.976160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1276/2541] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:13 2022] POST /identity/v3/projects => generated 365 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:14.020651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5abfc-48fb-4b5f-bdfe-93751002339b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:14.020651 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5abfc-48fb-4b5f-bdfe-93751002339b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:14.066257 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f3b01-5fc7-463a-ac1d-2b980940443f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:14.066974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f3b01-5fc7-463a-ac1d-2b980940443f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:14.067259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f3b01-5fc7-463a-ac1d-2b980940443f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:14.067527 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f3b01-5fc7-463a-ac1d-2b980940443f admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:14.068173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f3b01-5fc7-463a-ac1d-2b980940443f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:14.070466 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f3b01-5fc7-463a-ac1d-2b980940443f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:14.282196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1266/2542] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:13 2022] POST /identity/v3/users => generated 283 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:07:14.286466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb987cfa-efb1-43ee-9171-26bf8ff0dffc admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:14.299377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb987cfa-efb1-43ee-9171-26bf8ff0dffc admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:14.304065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36439dfa-9456-482b-b04c-f25343a07b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:14.304736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36439dfa-9456-482b-b04c-f25343a07b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:14.305053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36439dfa-9456-482b-b04c-f25343a07b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:14.305350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36439dfa-9456-482b-b04c-f25343a07b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:14.305959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36439dfa-9456-482b-b04c-f25343a07b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:14.307685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36439dfa-9456-482b-b04c-f25343a07b50 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:14.319432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1277/2543] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:14.323361 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f3b01-5fc7-463a-ac1d-2b980940443f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:14.337252 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f3b01-5fc7-463a-ac1d-2b980940443f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:14.343470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb56bc-bc85-472e-b556-b3ded09a5def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:14.344299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb56bc-bc85-472e-b556-b3ded09a5def admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:14.344612 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb56bc-bc85-472e-b556-b3ded09a5def admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:14.354484 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb56bc-bc85-472e-b556-b3ded09a5def admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:14.354484 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb56bc-bc85-472e-b556-b3ded09a5def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:14.354484 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb56bc-bc85-472e-b556-b3ded09a5def admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:14.362894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1267/2544] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:14.366455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36439dfa-9456-482b-b04c-f25343a07b50 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:14.378058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36439dfa-9456-482b-b04c-f25343a07b50 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:14.383211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dd3c6-9786-42c0-b0a8-2a5b1c519b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:14.383877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dd3c6-9786-42c0-b0a8-2a5b1c519b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:14.384160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dd3c6-9786-42c0-b0a8-2a5b1c519b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:14.384432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dd3c6-9786-42c0-b0a8-2a5b1c519b75 admin admin] PATH_INFO: `/v3/projects/7c43ccfea05e485c883c6d2ee57a6437/users/3fa4585b3fbc49a9bd1515374ca7f997/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:14.389086 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:14.400671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dd3c6-9786-42c0-b0a8-2a5b1c519b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c43ccfea05e485c883c6d2ee57a6437, user_id=3fa4585b3fbc49a9bd1515374ca7f997, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:14.404099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704dd3c6-9786-42c0-b0a8-2a5b1c519b75 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:14.453561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1278/2545] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:14 2022] PUT /identity/v3/projects/7c43ccfea05e485c883c6d2ee57a6437/users/3fa4585b3fbc49a9bd1515374ca7f997/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:14.465654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6639e1c-5a67-4b39-bb1d-ee977c46eabd None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:14.466512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6639e1c-5a67-4b39-bb1d-ee977c46eabd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:14.466512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6639e1c-5a67-4b39-bb1d-ee977c46eabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:14.498808 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c6639e1c-5a67-4b39-bb1d-ee977c46eabd None None] MFA Rules not processed for user `3fa4585b3fbc49a9bd1515374ca7f997`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:14.499958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c6639e1c-5a67-4b39-bb1d-ee977c46eabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:14.519788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:14.583849 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1268/2546] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:14.588741 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dd3c6-9786-42c0-b0a8-2a5b1c519b75 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:14.699855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dd3c6-9786-42c0-b0a8-2a5b1c519b75 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:14.705368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609aa631-f02e-474e-8376-0087a0e177c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:14.706163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609aa631-f02e-474e-8376-0087a0e177c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:14.706473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609aa631-f02e-474e-8376-0087a0e177c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:14.706773 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609aa631-f02e-474e-8376-0087a0e177c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:14.707491 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609aa631-f02e-474e-8376-0087a0e177c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:14.709533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609aa631-f02e-474e-8376-0087a0e177c5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:14.803649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1279/2547] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:14 2022] POST /identity/v3/projects => generated 365 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:14.808002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6639e1c-5a67-4b39-bb1d-ee977c46eabd None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:14.819998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6639e1c-5a67-4b39-bb1d-ee977c46eabd None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:14.887360 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2712f04-3bab-4c85-97b9-07d49747cca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:14.889668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2712f04-3bab-4c85-97b9-07d49747cca6 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:14.889668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2712f04-3bab-4c85-97b9-07d49747cca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:14.889925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2712f04-3bab-4c85-97b9-07d49747cca6 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:14.890447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2712f04-3bab-4c85-97b9-07d49747cca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:14.892428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2712f04-3bab-4c85-97b9-07d49747cca6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:15.021198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1269/2548] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:14 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:15.024507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609aa631-f02e-474e-8376-0087a0e177c5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:15.035891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609aa631-f02e-474e-8376-0087a0e177c5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:15.040564 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc509fa-fa77-4280-bc31-1163f5d689f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:15.042150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc509fa-fa77-4280-bc31-1163f5d689f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:15.042150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc509fa-fa77-4280-bc31-1163f5d689f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:15.042150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc509fa-fa77-4280-bc31-1163f5d689f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:15.042400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc509fa-fa77-4280-bc31-1163f5d689f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:15.043974 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc509fa-fa77-4280-bc31-1163f5d689f5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:15.055585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1280/2549] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:15.060287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2712f04-3bab-4c85-97b9-07d49747cca6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:15.071389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2712f04-3bab-4c85-97b9-07d49747cca6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:15.077184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db902f-e863-4d7c-a333-79d9e8521332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:15.077936 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db902f-e863-4d7c-a333-79d9e8521332 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:15.078283 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db902f-e863-4d7c-a333-79d9e8521332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:15.078684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db902f-e863-4d7c-a333-79d9e8521332 admin admin] PATH_INFO: `/v3/projects/27510e5e9e0f4b16947d15de9569f956/users/d1e3d10bc3704f33bc12f2c8453af4d9/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:15.096512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db902f-e863-4d7c-a333-79d9e8521332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27510e5e9e0f4b16947d15de9569f956, user_id=d1e3d10bc3704f33bc12f2c8453af4d9, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:15.099362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db902f-e863-4d7c-a333-79d9e8521332 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:15.174335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1270/2550] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:15 2022] PUT /identity/v3/projects/27510e5e9e0f4b16947d15de9569f956/users/d1e3d10bc3704f33bc12f2c8453af4d9/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:15.176145 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc509fa-fa77-4280-bc31-1163f5d689f5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:15.202715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc509fa-fa77-4280-bc31-1163f5d689f5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:15.279855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10c19b-e37b-4ffb-842e-ccee2fe8a821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:15.280465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10c19b-e37b-4ffb-842e-ccee2fe8a821 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:15.280743 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10c19b-e37b-4ffb-842e-ccee2fe8a821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:15.281037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10c19b-e37b-4ffb-842e-ccee2fe8a821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:15.281674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10c19b-e37b-4ffb-842e-ccee2fe8a821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:15.283262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10c19b-e37b-4ffb-842e-ccee2fe8a821 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:15.294922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1281/2551] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:15.298975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db902f-e863-4d7c-a333-79d9e8521332 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:15.310259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db902f-e863-4d7c-a333-79d9e8521332 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:15.314869 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21d77-516d-4df1-90fb-84d238ba9b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:15.315523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21d77-516d-4df1-90fb-84d238ba9b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:15.315817 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21d77-516d-4df1-90fb-84d238ba9b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:15.316071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b21d77-516d-4df1-90fb-84d238ba9b2c admin admin] PATH_INFO: `/v3/projects/27510e5e9e0f4b16947d15de9569f956/users/d1e3d10bc3704f33bc12f2c8453af4d9/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:15.318052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b21d77-516d-4df1-90fb-84d238ba9b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=27510e5e9e0f4b16947d15de9569f956, user_id=d1e3d10bc3704f33bc12f2c8453af4d9, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:15.320647 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b21d77-516d-4df1-90fb-84d238ba9b2c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:15.331653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1271/2552] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:15 2022] PUT /identity/v3/projects/27510e5e9e0f4b16947d15de9569f956/users/d1e3d10bc3704f33bc12f2c8453af4d9/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:15.341898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362f2c5-1510-40ed-b62e-369b1c0948e6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:15.342185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362f2c5-1510-40ed-b62e-369b1c0948e6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:15.342457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362f2c5-1510-40ed-b62e-369b1c0948e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:15.371455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a362f2c5-1510-40ed-b62e-369b1c0948e6 None None] MFA Rules not processed for user `d1e3d10bc3704f33bc12f2c8453af4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:15.373320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a362f2c5-1510-40ed-b62e-369b1c0948e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:15.426311 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:15.462156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1282/2553] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:15.470018 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21d77-516d-4df1-90fb-84d238ba9b2c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:15.471853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-12b21d77-516d-4df1-90fb-84d238ba9b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:15.538972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b21d77-516d-4df1-90fb-84d238ba9b2c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:15.546873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05debf38-fdf1-4a9b-9590-8b5b61a37ffd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:15.546873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05debf38-fdf1-4a9b-9590-8b5b61a37ffd service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:15.546873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05debf38-fdf1-4a9b-9590-8b5b61a37ffd service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:15.546873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05debf38-fdf1-4a9b-9590-8b5b61a37ffd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:15.558298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05debf38-fdf1-4a9b-9590-8b5b61a37ffd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:15.560535 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05debf38-fdf1-4a9b-9590-8b5b61a37ffd service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:15.572178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1272/2554] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:16.135407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362f2c5-1510-40ed-b62e-369b1c0948e6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:16.153891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362f2c5-1510-40ed-b62e-369b1c0948e6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:16.157965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed70c5-086e-44d8-8d98-270df79fa202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:16.158679 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed70c5-086e-44d8-8d98-270df79fa202 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:16.159060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed70c5-086e-44d8-8d98-270df79fa202 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:16.159428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed70c5-086e-44d8-8d98-270df79fa202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:16.226554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed70c5-086e-44d8-8d98-270df79fa202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:16.229674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed70c5-086e-44d8-8d98-270df79fa202 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:16.239956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1283/2555] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:20.156944 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05debf38-fdf1-4a9b-9590-8b5b61a37ffd service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:20.168164 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:20.231220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05debf38-fdf1-4a9b-9590-8b5b61a37ffd service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:20.243449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:20.244187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:20.244490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:20.244767 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] PATH_INFO: `/v3/users/3fa4585b3fbc49a9bd1515374ca7f997` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:20.245975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa4585b3fbc49a9bd1515374ca7f997)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:20.249246 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:20.434085 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa4585b3fbc49a9bd1515374ca7f997'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:20.435567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa4585b3fbc49a9bd1515374ca7f997'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:20.436336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa4585b3fbc49a9bd1515374ca7f997'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:20.525871 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa4585b3fbc49a9bd1515374ca7f997'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:20.539754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa4585b3fbc49a9bd1515374ca7f997'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:20.555803 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1273/2556] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:20 2022] DELETE /identity/v3/users/3fa4585b3fbc49a9bd1515374ca7f997 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:20.562958 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8628f2-cb2e-4c1a-8cfd-79a32bd15b99 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:20.567971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8628f2-cb2e-4c1a-8cfd-79a32bd15b99 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:20.567971 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8628f2-cb2e-4c1a-8cfd-79a32bd15b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:20.601421 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1b8628f2-cb2e-4c1a-8cfd-79a32bd15b99 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:20.602993 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1b8628f2-cb2e-4c1a-8cfd-79a32bd15b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:20.710089 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1284/2557] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:20.722653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:20.724113 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:20.805098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a600983-85e7-4dcb-89a4-44ee8ffac259 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:20.812030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e03ca0-dfef-4ef4-b4c0-896a1b7676dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:20.818149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e03ca0-dfef-4ef4-b4c0-896a1b7676dc service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:20.818149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e03ca0-dfef-4ef4-b4c0-896a1b7676dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:20.818149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e03ca0-dfef-4ef4-b4c0-896a1b7676dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:20.824745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e03ca0-dfef-4ef4-b4c0-896a1b7676dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:20.826519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e03ca0-dfef-4ef4-b4c0-896a1b7676dc service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:20.837523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1274/2558] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:21.431439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8628f2-cb2e-4c1a-8cfd-79a32bd15b99 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:21.433126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1b8628f2-cb2e-4c1a-8cfd-79a32bd15b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:21.447221 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8628f2-cb2e-4c1a-8cfd-79a32bd15b99 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:21.460793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:21.460793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:21.460793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:21.460793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] PATH_INFO: `/v3/projects/7c43ccfea05e485c883c6d2ee57a6437` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:21.461138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c43ccfea05e485c883c6d2ee57a6437)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:21.463797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:21.608519 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c43ccfea05e485c883c6d2ee57a6437', 'request_id': 'req-95a5f48c-4c49-44d7-be89-7cd4f4656abe'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:21.614403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c43ccfea05e485c883c6d2ee57a6437', 'request_id': 'req-95a5f48c-4c49-44d7-be89-7cd4f4656abe'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:21.621629 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c43ccfea05e485c883c6d2ee57a6437', 'request_id': 'req-95a5f48c-4c49-44d7-be89-7cd4f4656abe'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:21.692693 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] The token cache is being invalidate because project 7c43ccfea05e485c883c6d2ee57a6437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:21.692981 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:21.693635 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:21.695125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1285/2559] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:21 2022] DELETE /identity/v3/projects/7c43ccfea05e485c883c6d2ee57a6437 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:21.699343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e03ca0-dfef-4ef4-b4c0-896a1b7676dc service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:21.709554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e03ca0-dfef-4ef4-b4c0-896a1b7676dc service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:21.741069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:21.770411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:21.771081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:21.771431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:21.771708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] PATH_INFO: `/v3/users/d1e3d10bc3704f33bc12f2c8453af4d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:21.772857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1e3d10bc3704f33bc12f2c8453af4d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:21.774802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:22.035840 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1e3d10bc3704f33bc12f2c8453af4d9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.036650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1e3d10bc3704f33bc12f2c8453af4d9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.037470 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1e3d10bc3704f33bc12f2c8453af4d9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.109709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1e3d10bc3704f33bc12f2c8453af4d9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.123010 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1e3d10bc3704f33bc12f2c8453af4d9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.135597 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1275/2560] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:21 2022] DELETE /identity/v3/users/d1e3d10bc3704f33bc12f2c8453af4d9 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:22.341775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:22.343507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:22.351865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:22.357831 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5f48c-4c49-44d7-be89-7cd4f4656abe admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:22.420292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5b08e-4975-4100-ad6a-a3ac1e74f3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:22.420854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5b08e-4975-4100-ad6a-a3ac1e74f3c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:22.421160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5b08e-4975-4100-ad6a-a3ac1e74f3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:22.421435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5b08e-4975-4100-ad6a-a3ac1e74f3c0 admin admin] PATH_INFO: `/v3/users/a23a6f0be60647799828cd38bde5ebc9` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:22.422787 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5b08e-4975-4100-ad6a-a3ac1e74f3c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a23a6f0be60647799828cd38bde5ebc9)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:22.425203 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5b08e-4975-4100-ad6a-a3ac1e74f3c0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:22.534719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:22.543617 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:22.550107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3173d-0235-4c9a-954e-93299c10623b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:22.554980 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:22.555900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:22.556220 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:22.556528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] PATH_INFO: `/v3/projects/27510e5e9e0f4b16947d15de9569f956` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:22.557702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27510e5e9e0f4b16947d15de9569f956)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:22.560363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:22.599485 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ddc5b08e-4975-4100-ad6a-a3ac1e74f3c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a23a6f0be60647799828cd38bde5ebc9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.606909 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ddc5b08e-4975-4100-ad6a-a3ac1e74f3c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a23a6f0be60647799828cd38bde5ebc9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.688273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27510e5e9e0f4b16947d15de9569f956', 'request_id': 'req-0054d462-72e2-4574-84c7-411cfb7eb0f7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.692415 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ddc5b08e-4975-4100-ad6a-a3ac1e74f3c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a23a6f0be60647799828cd38bde5ebc9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.693063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ddc5b08e-4975-4100-ad6a-a3ac1e74f3c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a23a6f0be60647799828cd38bde5ebc9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.694467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27510e5e9e0f4b16947d15de9569f956', 'request_id': 'req-0054d462-72e2-4574-84c7-411cfb7eb0f7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.701197 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27510e5e9e0f4b16947d15de9569f956', 'request_id': 'req-0054d462-72e2-4574-84c7-411cfb7eb0f7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.707254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ddc5b08e-4975-4100-ad6a-a3ac1e74f3c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a23a6f0be60647799828cd38bde5ebc9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.713236 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1286/2561] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:22 2022] DELETE /identity/v3/users/a23a6f0be60647799828cd38bde5ebc9 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:22.717531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19722d9-4345-4ec7-b8c3-0e7e78a2dacd None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:22.717820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19722d9-4345-4ec7-b8c3-0e7e78a2dacd None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:22.718352 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19722d9-4345-4ec7-b8c3-0e7e78a2dacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:22.737219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e19722d9-4345-4ec7-b8c3-0e7e78a2dacd None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:22.738425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e19722d9-4345-4ec7-b8c3-0e7e78a2dacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:22.772207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] The token cache is being invalidate because project 27510e5e9e0f4b16947d15de9569f956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:22.772468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.773128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0054d462-72e2-4574-84c7-411cfb7eb0f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:22.780536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1276/2562] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:22 2022] DELETE /identity/v3/projects/27510e5e9e0f4b16947d15de9569f956 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:22.782295 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b24009-d1c6-43c3-b952-ec28e2517752 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:22.782295 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b24009-d1c6-43c3-b952-ec28e2517752 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:22.782644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b24009-d1c6-43c3-b952-ec28e2517752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:22.802478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a1b24009-d1c6-43c3-b952-ec28e2517752 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:22.803708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a1b24009-d1c6-43c3-b952-ec28e2517752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:22.804065 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1287/2563] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:07:22.812311 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1277/2564] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:07:22.818294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19722d9-4345-4ec7-b8c3-0e7e78a2dacd None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:22.819712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff572a2-df76-4e60-bfef-1537df4eda43 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:22.819900 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e19722d9-4345-4ec7-b8c3-0e7e78a2dacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:22.820005 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff572a2-df76-4e60-bfef-1537df4eda43 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:22.820255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff572a2-df76-4e60-bfef-1537df4eda43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:22.839508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6ff572a2-df76-4e60-bfef-1537df4eda43 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:22.841689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6ff572a2-df76-4e60-bfef-1537df4eda43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:22.847784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:22.849966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1278/2565] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:07:22.854127 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff572a2-df76-4e60-bfef-1537df4eda43 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:22.863652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff572a2-df76-4e60-bfef-1537df4eda43 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:22.868258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc3fda-00ca-4f75-bab3-45288f451e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:22.868846 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc3fda-00ca-4f75-bab3-45288f451e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:22.869161 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc3fda-00ca-4f75-bab3-45288f451e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:22.869430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc3fda-00ca-4f75-bab3-45288f451e14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:22.870064 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc3fda-00ca-4f75-bab3-45288f451e14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:07:22.870362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc3fda-00ca-4f75-bab3-45288f451e14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:22.871938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc3fda-00ca-4f75-bab3-45288f451e14 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:22.879805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19722d9-4345-4ec7-b8c3-0e7e78a2dacd None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:22.883620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ee286-00dc-4322-94b3-e2a7fa9e23b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:22.883961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1279/2566] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:07:22.884225 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee286-00dc-4322-94b3-e2a7fa9e23b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:22.884505 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee286-00dc-4322-94b3-e2a7fa9e23b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:22.884853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ee286-00dc-4322-94b3-e2a7fa9e23b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:22.888308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc3fda-00ca-4f75-bab3-45288f451e14 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:22.895660 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ee286-00dc-4322-94b3-e2a7fa9e23b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:22.899078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ee286-00dc-4322-94b3-e2a7fa9e23b4 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:22.899186 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc3fda-00ca-4f75-bab3-45288f451e14 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:22.904042 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ad401-c532-444e-b330-f65d18081ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:22.904814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ad401-c532-444e-b330-f65d18081ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:22.905139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ad401-c532-444e-b330-f65d18081ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:22.905473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ad401-c532-444e-b330-f65d18081ce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:22.906339 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ad401-c532-444e-b330-f65d18081ce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:22.907000 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1288/2567] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:22.908247 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ad401-c532-444e-b330-f65d18081ce9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:22.913674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:22.986364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1280/2568] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:22 2022] POST /identity/v3/projects => generated 393 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:22.990447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ee286-00dc-4322-94b3-e2a7fa9e23b4 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:23.001357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ee286-00dc-4322-94b3-e2a7fa9e23b4 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:23.060909 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775c993-c402-4003-9daa-faab767e219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:23.061566 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775c993-c402-4003-9daa-faab767e219e admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:23.061847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775c993-c402-4003-9daa-faab767e219e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:23.062117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775c993-c402-4003-9daa-faab767e219e admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:23.062756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775c993-c402-4003-9daa-faab767e219e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:23.064753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775c993-c402-4003-9daa-faab767e219e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:23.135408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1289/2569] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:22 2022] POST /identity/v3/users => generated 297 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:07:23.137322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ad401-c532-444e-b330-f65d18081ce9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:23.162031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ad401-c532-444e-b330-f65d18081ce9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:23.174497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe39a2-baf6-4f7a-ad18-de391cea5d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:23.174497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe39a2-baf6-4f7a-ad18-de391cea5d99 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:23.174497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe39a2-baf6-4f7a-ad18-de391cea5d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:23.174497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe39a2-baf6-4f7a-ad18-de391cea5d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:23.175375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe39a2-baf6-4f7a-ad18-de391cea5d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:23.175715 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe39a2-baf6-4f7a-ad18-de391cea5d99 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:23.211652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1281/2570] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:23.212739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775c993-c402-4003-9daa-faab767e219e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:23.225377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775c993-c402-4003-9daa-faab767e219e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:23.232277 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3ef8d-86d8-4b8b-a6a2-9541cc9c7157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:23.233116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3ef8d-86d8-4b8b-a6a2-9541cc9c7157 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:23.233398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3ef8d-86d8-4b8b-a6a2-9541cc9c7157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:23.233936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3ef8d-86d8-4b8b-a6a2-9541cc9c7157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:23.234320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3ef8d-86d8-4b8b-a6a2-9541cc9c7157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:23.235917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3ef8d-86d8-4b8b-a6a2-9541cc9c7157 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:23.251506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1290/2571] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:23.255322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe39a2-baf6-4f7a-ad18-de391cea5d99 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:23.266272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe39a2-baf6-4f7a-ad18-de391cea5d99 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:23.271110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfee1d-8208-497f-819f-cb4326d2b0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:23.272103 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfee1d-8208-497f-819f-cb4326d2b0a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:23.272103 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfee1d-8208-497f-819f-cb4326d2b0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:23.272469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfee1d-8208-497f-819f-cb4326d2b0a1 admin admin] PATH_INFO: `/v3/projects/e504b99be3a84305a6c55fac00bf0086/users/1928976dcb3f4b58875a534d0365b12f/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:23.289469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfee1d-8208-497f-819f-cb4326d2b0a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e504b99be3a84305a6c55fac00bf0086, user_id=1928976dcb3f4b58875a534d0365b12f, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:23.292328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfee1d-8208-497f-819f-cb4326d2b0a1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:23.385548 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3ef8d-86d8-4b8b-a6a2-9541cc9c7157 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:23.387449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a7e3ef8d-86d8-4b8b-a6a2-9541cc9c7157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:23.401968 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3ef8d-86d8-4b8b-a6a2-9541cc9c7157 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:23.409145 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1282/2572] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:23 2022] PUT /identity/v3/projects/e504b99be3a84305a6c55fac00bf0086/users/1928976dcb3f4b58875a534d0365b12f/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:23.425054 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c49f1-f3ef-43ff-b089-ecddb50dfa82 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:23.425279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c49f1-f3ef-43ff-b089-ecddb50dfa82 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:23.425449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c49f1-f3ef-43ff-b089-ecddb50dfa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:23.456243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-826c49f1-f3ef-43ff-b089-ecddb50dfa82 None None] MFA Rules not processed for user `1928976dcb3f4b58875a534d0365b12f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:23.457710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-826c49f1-f3ef-43ff-b089-ecddb50dfa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:23.476983 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:23.477657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:23.477985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:23.478340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] PATH_INFO: `/v3/projects/e19c257e01a04a4da046089ed09e61b7` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:23.482090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19c257e01a04a4da046089ed09e61b7)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:23.482911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:23.535362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1283/2573] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:07:23.540016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c49f1-f3ef-43ff-b089-ecddb50dfa82 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:23.551423 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c49f1-f3ef-43ff-b089-ecddb50dfa82 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:23.556057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc898e-42a9-4773-b122-4da0461f9570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:23.556649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc898e-42a9-4773-b122-4da0461f9570 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:23.557092 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc898e-42a9-4773-b122-4da0461f9570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:23.557248 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc898e-42a9-4773-b122-4da0461f9570 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:23.557871 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc898e-42a9-4773-b122-4da0461f9570 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:23.559864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc898e-42a9-4773-b122-4da0461f9570 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:23.599547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:23.602510 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19c257e01a04a4da046089ed09e61b7', 'request_id': 'req-8ac3d82d-112f-4f54-96da-858941deb16e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:23.608625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19c257e01a04a4da046089ed09e61b7', 'request_id': 'req-8ac3d82d-112f-4f54-96da-858941deb16e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:23.615256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19c257e01a04a4da046089ed09e61b7', 'request_id': 'req-8ac3d82d-112f-4f54-96da-858941deb16e'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:23.670678 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1284/2574] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:23 2022] POST /identity/v3/projects => generated 391 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:23.674898 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc898e-42a9-4773-b122-4da0461f9570 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:23.696754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] The token cache is being invalidate because project e19c257e01a04a4da046089ed09e61b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:23.706877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:23.706877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:23.706877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1291/2575] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:23 2022] DELETE /identity/v3/projects/e19c257e01a04a4da046089ed09e61b7 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:23.708368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:23.709803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:23.739806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc898e-42a9-4773-b122-4da0461f9570 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:23.746747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7dcd4-a920-4847-98fd-b3bfcc468ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:23.746747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7dcd4-a920-4847-98fd-b3bfcc468ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:23.746747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7dcd4-a920-4847-98fd-b3bfcc468ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:23.746747 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7dcd4-a920-4847-98fd-b3bfcc468ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:23.747455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7dcd4-a920-4847-98fd-b3bfcc468ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:23.749472 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7dcd4-a920-4847-98fd-b3bfcc468ee3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:23.776029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3d82d-112f-4f54-96da-858941deb16e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:23.781908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:23.782545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:23.782835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:23.783111 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] PATH_INFO: `/v3/users/b7d7994856c54d989ccc8510a73c2477` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:23.784259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d7994856c54d989ccc8510a73c2477)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:23.786219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:23.824428 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1285/2576] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:23 2022] POST /identity/v3/users => generated 302 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:07:23.828948 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7dcd4-a920-4847-98fd-b3bfcc468ee3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:23.830595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-aee7dcd4-a920-4847-98fd-b3bfcc468ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:23.844974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7dcd4-a920-4847-98fd-b3bfcc468ee3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:23.849722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34ff9e-15b4-4376-971b-9c4fce00022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:23.850359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34ff9e-15b4-4376-971b-9c4fce00022d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:23.850638 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34ff9e-15b4-4376-971b-9c4fce00022d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:23.850910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34ff9e-15b4-4376-971b-9c4fce00022d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:23.851585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34ff9e-15b4-4376-971b-9c4fce00022d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:23.853208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34ff9e-15b4-4376-971b-9c4fce00022d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:23.854545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:23.863968 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1286/2577] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:23.867986 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34ff9e-15b4-4376-971b-9c4fce00022d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:23.878166 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34ff9e-15b4-4376-971b-9c4fce00022d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:23.883099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c568-e632-4bf3-9639-c88a39fe52c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:23.883720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c568-e632-4bf3-9639-c88a39fe52c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:23.883998 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c568-e632-4bf3-9639-c88a39fe52c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:23.884642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641c568-e632-4bf3-9639-c88a39fe52c4 admin admin] PATH_INFO: `/v3/projects/6133c4b9d21f4527b92b068d451ad272/users/aa0347a1ac8b4c49a647f064f9d901f6/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:23.899587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641c568-e632-4bf3-9639-c88a39fe52c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6133c4b9d21f4527b92b068d451ad272, user_id=aa0347a1ac8b4c49a647f064f9d901f6, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:23.902356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641c568-e632-4bf3-9639-c88a39fe52c4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:23.975197 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d7994856c54d989ccc8510a73c2477'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:23.982674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d7994856c54d989ccc8510a73c2477'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:24.041468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1287/2578] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:23 2022] PUT /identity/v3/projects/6133c4b9d21f4527b92b068d451ad272/users/aa0347a1ac8b4c49a647f064f9d901f6/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:24.046303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c568-e632-4bf3-9639-c88a39fe52c4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:24.058460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d7994856c54d989ccc8510a73c2477'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:24.059109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d7994856c54d989ccc8510a73c2477'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:24.071977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d7994856c54d989ccc8510a73c2477'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:24.078184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1292/2579] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:23 2022] DELETE /identity/v3/users/b7d7994856c54d989ccc8510a73c2477 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:24.108684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641c568-e632-4bf3-9639-c88a39fe52c4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:24.166751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415bb98-005d-44db-9368-bb8ecb2315ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:24.167632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415bb98-005d-44db-9368-bb8ecb2315ab admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:24.167909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415bb98-005d-44db-9368-bb8ecb2315ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:24.168197 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415bb98-005d-44db-9368-bb8ecb2315ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:24.169362 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415bb98-005d-44db-9368-bb8ecb2315ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:24.171027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415bb98-005d-44db-9368-bb8ecb2315ab admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:24.180966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1288/2580] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:07:24.185716 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:24.187241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:24.201483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648ed36-9d3b-4969-b0ae-484533c2596c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:24.206048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c04748-9c69-48fe-8ca9-7b0fdab4210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:24.206697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c04748-9c69-48fe-8ca9-7b0fdab4210b admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:24.206979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c04748-9c69-48fe-8ca9-7b0fdab4210b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:24.207254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c04748-9c69-48fe-8ca9-7b0fdab4210b admin admin] PATH_INFO: `/v3/projects/6133c4b9d21f4527b92b068d451ad272/users/aa0347a1ac8b4c49a647f064f9d901f6/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:24.209165 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c04748-9c69-48fe-8ca9-7b0fdab4210b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6133c4b9d21f4527b92b068d451ad272, user_id=aa0347a1ac8b4c49a647f064f9d901f6, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:24.211799 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c04748-9c69-48fe-8ca9-7b0fdab4210b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:24.223174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1293/2581] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:24 2022] PUT /identity/v3/projects/6133c4b9d21f4527b92b068d451ad272/users/aa0347a1ac8b4c49a647f064f9d901f6/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:24.234206 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813b838-eac0-49a7-8725-3b65f1fe2834 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:24.234496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813b838-eac0-49a7-8725-3b65f1fe2834 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:24.234839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813b838-eac0-49a7-8725-3b65f1fe2834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:24.267660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4813b838-eac0-49a7-8725-3b65f1fe2834 None None] MFA Rules not processed for user `aa0347a1ac8b4c49a647f064f9d901f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:24.268899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4813b838-eac0-49a7-8725-3b65f1fe2834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:24.377121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1289/2582] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:24 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:24.391940 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c04748-9c69-48fe-8ca9-7b0fdab4210b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:24.394905 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d8c04748-9c69-48fe-8ca9-7b0fdab4210b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:24.447505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:24.499818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:24.514137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c04748-9c69-48fe-8ca9-7b0fdab4210b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:24.518151 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf740c18-01ff-46c8-b093-ec50a8c9a50c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:24.523600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf740c18-01ff-46c8-b093-ec50a8c9a50c service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:24.523888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf740c18-01ff-46c8-b093-ec50a8c9a50c service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:24.524321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf740c18-01ff-46c8-b093-ec50a8c9a50c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:24.536312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf740c18-01ff-46c8-b093-ec50a8c9a50c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:24.537991 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf740c18-01ff-46c8-b093-ec50a8c9a50c service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:24.549918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1294/2583] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:24 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:24.581728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813b838-eac0-49a7-8725-3b65f1fe2834 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:24.583069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4813b838-eac0-49a7-8725-3b65f1fe2834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:24.639253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf740c18-01ff-46c8-b093-ec50a8c9a50c service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:24.640668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cf740c18-01ff-46c8-b093-ec50a8c9a50c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:24.657929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf740c18-01ff-46c8-b093-ec50a8c9a50c service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:24.675662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e37b83-12c0-4c50-aa78-9f3a29811338 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:24.677391 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e37b83-12c0-4c50-aa78-9f3a29811338 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:24.677678 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e37b83-12c0-4c50-aa78-9f3a29811338 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:24.677951 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e37b83-12c0-4c50-aa78-9f3a29811338 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:24.683647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-14e37b83-12c0-4c50-aa78-9f3a29811338 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:24.702832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813b838-eac0-49a7-8725-3b65f1fe2834 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:24.707867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:24.708710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:24.708996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:24.709313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] PATH_INFO: `/v3/projects/a18932efd5724e1098f75daf3802833c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:24.710451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a18932efd5724e1098f75daf3802833c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:24.712381 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:24.765713 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e37b83-12c0-4c50-aa78-9f3a29811338 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:24.767442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e37b83-12c0-4c50-aa78-9f3a29811338 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:24.776574 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1295/2584] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:07:24.864779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a18932efd5724e1098f75daf3802833c', 'request_id': 'req-bc185412-84f8-4ac5-8b74-7a943a75860b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:24.871128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a18932efd5724e1098f75daf3802833c', 'request_id': 'req-bc185412-84f8-4ac5-8b74-7a943a75860b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:24.878523 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a18932efd5724e1098f75daf3802833c', 'request_id': 'req-bc185412-84f8-4ac5-8b74-7a943a75860b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:24.955699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] The token cache is being invalidate because project a18932efd5724e1098f75daf3802833c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:24.956148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:24.956996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-bc185412-84f8-4ac5-8b74-7a943a75860b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:24.958550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1290/2585] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:24 2022] DELETE /identity/v3/projects/a18932efd5724e1098f75daf3802833c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:24.963791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e0d16-1e1f-4466-8d1e-35ebeb4de288 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:24.963791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e0d16-1e1f-4466-8d1e-35ebeb4de288 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:24.964001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e0d16-1e1f-4466-8d1e-35ebeb4de288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:24.984138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4d1e0d16-1e1f-4466-8d1e-35ebeb4de288 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:24.991360 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4d1e0d16-1e1f-4466-8d1e-35ebeb4de288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:25.050332 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1296/2586] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:07:25.058343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291a62e-3897-4ed1-bc86-4c1c5d15fef8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:25.058633 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291a62e-3897-4ed1-bc86-4c1c5d15fef8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:25.058915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291a62e-3897-4ed1-bc86-4c1c5d15fef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:25.079893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0291a62e-3897-4ed1-bc86-4c1c5d15fef8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:25.081087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0291a62e-3897-4ed1-bc86-4c1c5d15fef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:25.089399 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1291/2587] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:25.094747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e0d16-1e1f-4466-8d1e-35ebeb4de288 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:25.105410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e0d16-1e1f-4466-8d1e-35ebeb4de288 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:25.111067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f73e6-2460-43b4-a3aa-be7ad620be03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:25.111690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f73e6-2460-43b4-a3aa-be7ad620be03 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:25.111970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f73e6-2460-43b4-a3aa-be7ad620be03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:25.112240 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f73e6-2460-43b4-a3aa-be7ad620be03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:25.112898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f73e6-2460-43b4-a3aa-be7ad620be03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:07:25.113243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f73e6-2460-43b4-a3aa-be7ad620be03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:25.114844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f73e6-2460-43b4-a3aa-be7ad620be03 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:25.134547 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291a62e-3897-4ed1-bc86-4c1c5d15fef8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:25.136327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1297/2588] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:07:25.150133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291a62e-3897-4ed1-bc86-4c1c5d15fef8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:25.152851 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ac554-1600-4ced-a5bf-d1ac38b2df46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:25.154296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ac554-1600-4ced-a5bf-d1ac38b2df46 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:25.154870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ac554-1600-4ced-a5bf-d1ac38b2df46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:25.155125 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ac554-1600-4ced-a5bf-d1ac38b2df46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:25.155925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ac554-1600-4ced-a5bf-d1ac38b2df46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:25.158369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ac554-1600-4ced-a5bf-d1ac38b2df46 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:25.260171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1292/2589] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:25 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:07:25.264580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f73e6-2460-43b4-a3aa-be7ad620be03 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:25.276474 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f73e6-2460-43b4-a3aa-be7ad620be03 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:25.335496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d7a51-a50e-4c39-b4aa-caed67ab8781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:25.336161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d7a51-a50e-4c39-b4aa-caed67ab8781 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:25.336398 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d7a51-a50e-4c39-b4aa-caed67ab8781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:25.336671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d7a51-a50e-4c39-b4aa-caed67ab8781 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:25.337347 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d7a51-a50e-4c39-b4aa-caed67ab8781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:25.339259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d7a51-a50e-4c39-b4aa-caed67ab8781 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:25.435902 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:25.442172 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1298/2590] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:25 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:25.446367 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ac554-1600-4ced-a5bf-d1ac38b2df46 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:25.462654 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ac554-1600-4ced-a5bf-d1ac38b2df46 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:25.473965 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831726a-54c9-4a9a-8ab7-b3e18791a0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:25.474595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831726a-54c9-4a9a-8ab7-b3e18791a0be admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:25.474877 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831726a-54c9-4a9a-8ab7-b3e18791a0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:25.475165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831726a-54c9-4a9a-8ab7-b3e18791a0be admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:25.475845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831726a-54c9-4a9a-8ab7-b3e18791a0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:25.477447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831726a-54c9-4a9a-8ab7-b3e18791a0be admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:25.498944 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1293/2591] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:25.503493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d7a51-a50e-4c39-b4aa-caed67ab8781 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:25.516308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d7a51-a50e-4c39-b4aa-caed67ab8781 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:25.522280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66907a-7768-4d80-8a20-20910d370a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:25.523278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66907a-7768-4d80-8a20-20910d370a77 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:25.523572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66907a-7768-4d80-8a20-20910d370a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:25.523850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66907a-7768-4d80-8a20-20910d370a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:25.524506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66907a-7768-4d80-8a20-20910d370a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:25.526150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66907a-7768-4d80-8a20-20910d370a77 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:25.548256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1299/2592] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:07:25.553342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831726a-54c9-4a9a-8ab7-b3e18791a0be admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:25.564517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831726a-54c9-4a9a-8ab7-b3e18791a0be admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:25.579730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3540e5-2959-48c5-8d36-a22ba16aa5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:25.580707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3540e5-2959-48c5-8d36-a22ba16aa5ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:25.581318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3540e5-2959-48c5-8d36-a22ba16aa5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:25.581729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3540e5-2959-48c5-8d36-a22ba16aa5ba admin admin] PATH_INFO: `/v3/projects/ad093a10d59d4bbdacb896101cdb62d3/users/ef4debe7f7f3494ab69c761516dd2980/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:25.604717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3540e5-2959-48c5-8d36-a22ba16aa5ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad093a10d59d4bbdacb896101cdb62d3, user_id=ef4debe7f7f3494ab69c761516dd2980, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:25.607100 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3540e5-2959-48c5-8d36-a22ba16aa5ba admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:25.744564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1294/2593] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:25 2022] PUT /identity/v3/projects/ad093a10d59d4bbdacb896101cdb62d3/users/ef4debe7f7f3494ab69c761516dd2980/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:25.755500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61727eb6-2f75-4158-9247-c492d645eafc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:25.755788 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61727eb6-2f75-4158-9247-c492d645eafc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:25.756084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61727eb6-2f75-4158-9247-c492d645eafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:25.783608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-61727eb6-2f75-4158-9247-c492d645eafc None None] MFA Rules not processed for user `ef4debe7f7f3494ab69c761516dd2980`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:25.784810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-61727eb6-2f75-4158-9247-c492d645eafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:25.849562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1300/2594] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:25.854039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3540e5-2959-48c5-8d36-a22ba16aa5ba admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:25.885722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:25.919344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3540e5-2959-48c5-8d36-a22ba16aa5ba admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:25.924283 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5c783-c868-4ed3-b2be-8e812c41d265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:25.924978 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c783-c868-4ed3-b2be-8e812c41d265 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:25.925347 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c783-c868-4ed3-b2be-8e812c41d265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:25.925625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c783-c868-4ed3-b2be-8e812c41d265 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:25.926267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5c783-c868-4ed3-b2be-8e812c41d265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:25.928199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5c783-c868-4ed3-b2be-8e812c41d265 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:26.023811 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1295/2595] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:25 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:26.024858 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61727eb6-2f75-4158-9247-c492d645eafc None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:26.035616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61727eb6-2f75-4158-9247-c492d645eafc None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:26.094323 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39cdb2-ff22-457c-b06a-d6333611d99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:26.095040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39cdb2-ff22-457c-b06a-d6333611d99a admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:26.095324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39cdb2-ff22-457c-b06a-d6333611d99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:26.095599 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39cdb2-ff22-457c-b06a-d6333611d99a admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:26.096242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39cdb2-ff22-457c-b06a-d6333611d99a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:26.098192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39cdb2-ff22-457c-b06a-d6333611d99a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:26.209827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1301/2596] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:26 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:26.212135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5c783-c868-4ed3-b2be-8e812c41d265 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:26.223464 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5c783-c868-4ed3-b2be-8e812c41d265 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:26.228427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49e466-d98c-48f8-a5c3-35c5534ed673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:26.229144 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49e466-d98c-48f8-a5c3-35c5534ed673 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:26.229497 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49e466-d98c-48f8-a5c3-35c5534ed673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:26.229866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49e466-d98c-48f8-a5c3-35c5534ed673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:26.230586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49e466-d98c-48f8-a5c3-35c5534ed673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:26.232839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49e466-d98c-48f8-a5c3-35c5534ed673 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:26.243574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1296/2597] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:07:26.248192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39cdb2-ff22-457c-b06a-d6333611d99a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:26.258822 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39cdb2-ff22-457c-b06a-d6333611d99a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:26.264232 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fd359-bd8b-44cf-ae42-da6715c1a2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:26.271683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fd359-bd8b-44cf-ae42-da6715c1a2e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:26.271683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fd359-bd8b-44cf-ae42-da6715c1a2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:26.271683 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fd359-bd8b-44cf-ae42-da6715c1a2e1 admin admin] PATH_INFO: `/v3/projects/202dbaf7927441a29261ba4fd2741c66/users/e8dd6704ce5f4a6ca8a8a2847b8f176d/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:26.281039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fd359-bd8b-44cf-ae42-da6715c1a2e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=202dbaf7927441a29261ba4fd2741c66, user_id=e8dd6704ce5f4a6ca8a8a2847b8f176d, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:26.283914 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fd359-bd8b-44cf-ae42-da6715c1a2e1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:26.349570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1302/2598] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:26 2022] PUT /identity/v3/projects/202dbaf7927441a29261ba4fd2741c66/users/e8dd6704ce5f4a6ca8a8a2847b8f176d/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:26.353551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49e466-d98c-48f8-a5c3-35c5534ed673 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:26.364459 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49e466-d98c-48f8-a5c3-35c5534ed673 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:26.421868 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecc941-fb76-421f-bb79-05891dfbf52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:26.422478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecc941-fb76-421f-bb79-05891dfbf52b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:26.422753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecc941-fb76-421f-bb79-05891dfbf52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:26.423023 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecc941-fb76-421f-bb79-05891dfbf52b admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:26.423667 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecc941-fb76-421f-bb79-05891dfbf52b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:26.425318 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecc941-fb76-421f-bb79-05891dfbf52b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:26.438124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1297/2599] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:26.446845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fd359-bd8b-44cf-ae42-da6715c1a2e1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:26.458183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fd359-bd8b-44cf-ae42-da6715c1a2e1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:26.463053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f30fc-2253-4788-b2b5-28a18406ad67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:26.463695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f30fc-2253-4788-b2b5-28a18406ad67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:26.464118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f30fc-2253-4788-b2b5-28a18406ad67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:26.464396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f30fc-2253-4788-b2b5-28a18406ad67 admin admin] PATH_INFO: `/v3/projects/202dbaf7927441a29261ba4fd2741c66/users/e8dd6704ce5f4a6ca8a8a2847b8f176d/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:26.466545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f30fc-2253-4788-b2b5-28a18406ad67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=202dbaf7927441a29261ba4fd2741c66, user_id=e8dd6704ce5f4a6ca8a8a2847b8f176d, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:26.469767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f30fc-2253-4788-b2b5-28a18406ad67 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:26.482554 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1303/2600] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:26 2022] PUT /identity/v3/projects/202dbaf7927441a29261ba4fd2741c66/users/e8dd6704ce5f4a6ca8a8a2847b8f176d/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:26.493777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb59e33-b6e6-4993-9dcf-0da884ef5817 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:26.494077 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb59e33-b6e6-4993-9dcf-0da884ef5817 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:26.494378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb59e33-b6e6-4993-9dcf-0da884ef5817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:26.524837 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ceb59e33-b6e6-4993-9dcf-0da884ef5817 None None] MFA Rules not processed for user `e8dd6704ce5f4a6ca8a8a2847b8f176d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:26.526276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ceb59e33-b6e6-4993-9dcf-0da884ef5817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:26.601810 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1298/2601] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:26.624925 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f30fc-2253-4788-b2b5-28a18406ad67 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:26.624925 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-503f30fc-2253-4788-b2b5-28a18406ad67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:26.657751 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:26.688237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f30fc-2253-4788-b2b5-28a18406ad67 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:26.692251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e2447-604f-4541-882f-93fc121357dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:26.692942 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e2447-604f-4541-882f-93fc121357dc service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:26.693315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e2447-604f-4541-882f-93fc121357dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:26.693652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e2447-604f-4541-882f-93fc121357dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:26.704868 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e2447-604f-4541-882f-93fc121357dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:26.706451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e2447-604f-4541-882f-93fc121357dc service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:26.720382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1304/2602] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:27.118908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb59e33-b6e6-4993-9dcf-0da884ef5817 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:27.129072 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb59e33-b6e6-4993-9dcf-0da884ef5817 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:27.132809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f20913-c8b1-48f3-a073-b3bf01926cb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:27.133771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f20913-c8b1-48f3-a073-b3bf01926cb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:27.133954 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f20913-c8b1-48f3-a073-b3bf01926cb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:27.134228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f20913-c8b1-48f3-a073-b3bf01926cb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:27.158520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:27.194754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f20913-c8b1-48f3-a073-b3bf01926cb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:27.196177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f20913-c8b1-48f3-a073-b3bf01926cb7 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:27.206003 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1299/2603] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:29.092571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25a54f-8c62-40db-a4a1-5f047e07ff07 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:29.093433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25a54f-8c62-40db-a4a1-5f047e07ff07 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:29.093783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25a54f-8c62-40db-a4a1-5f047e07ff07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:29.120435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1c25a54f-8c62-40db-a4a1-5f047e07ff07 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:29.121766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1c25a54f-8c62-40db-a4a1-5f047e07ff07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:29.199370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1305/2604] 10.222.0.17 () {58 vars in 967 bytes} [Thu Aug 11 10:07:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:07:29.213531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f20913-c8b1-48f3-a073-b3bf01926cb7 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:29.215592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-01f20913-c8b1-48f3-a073-b3bf01926cb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:29.305454 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f20913-c8b1-48f3-a073-b3bf01926cb7 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:29.310809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bfd0e-1090-49e3-a0bf-d863aa482a25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:29.314692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bfd0e-1090-49e3-a0bf-d863aa482a25 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:29.314692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bfd0e-1090-49e3-a0bf-d863aa482a25 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:29.314692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bfd0e-1090-49e3-a0bf-d863aa482a25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:29.321223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bfd0e-1090-49e3-a0bf-d863aa482a25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:29.322798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bfd0e-1090-49e3-a0bf-d863aa482a25 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:29.331703 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1300/2605] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:07:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:07:30.154275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25a54f-8c62-40db-a4a1-5f047e07ff07 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:30.156062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1c25a54f-8c62-40db-a4a1-5f047e07ff07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:30.233722 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25a54f-8c62-40db-a4a1-5f047e07ff07 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:30.242989 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:30.243766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:30.244137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:30.244417 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] PATH_INFO: `/v3/users/1928976dcb3f4b58875a534d0365b12f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:30.246125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1928976dcb3f4b58875a534d0365b12f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:30.248317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:30.403934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1928976dcb3f4b58875a534d0365b12f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:30.411397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1928976dcb3f4b58875a534d0365b12f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:30.469771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1928976dcb3f4b58875a534d0365b12f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:30.470386 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1928976dcb3f4b58875a534d0365b12f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:30.483666 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:30.484992 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1928976dcb3f4b58875a534d0365b12f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:30.490830 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1306/2606] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:30 2022] DELETE /identity/v3/users/1928976dcb3f4b58875a534d0365b12f => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:30.495304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d688c9-7c6b-48ae-a4aa-0aacef55cbd0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:30.496136 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d688c9-7c6b-48ae-a4aa-0aacef55cbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:30.496418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d688c9-7c6b-48ae-a4aa-0aacef55cbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:30.515625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-58d688c9-7c6b-48ae-a4aa-0aacef55cbd0 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:30.516846 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-58d688c9-7c6b-48ae-a4aa-0aacef55cbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:30.577187 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1301/2607] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:30.587182 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:30.588622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:30.646740 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab39106-06e2-4880-afa3-6c034045d3d1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:30.650331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185903d3-8aa2-4715-b517-441f5b86d253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:30.650937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185903d3-8aa2-4715-b517-441f5b86d253 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:30.651212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185903d3-8aa2-4715-b517-441f5b86d253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:30.651483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185903d3-8aa2-4715-b517-441f5b86d253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:30.661033 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185903d3-8aa2-4715-b517-441f5b86d253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:30.662451 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185903d3-8aa2-4715-b517-441f5b86d253 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:30.671659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1307/2608] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:31.083224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d688c9-7c6b-48ae-a4aa-0aacef55cbd0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:31.084632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-58d688c9-7c6b-48ae-a4aa-0aacef55cbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:31.098254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d688c9-7c6b-48ae-a4aa-0aacef55cbd0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:31.103150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:31.103986 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:31.104376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:31.108293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] PATH_INFO: `/v3/projects/e504b99be3a84305a6c55fac00bf0086` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:31.108293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e504b99be3a84305a6c55fac00bf0086)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:31.110554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:31.215578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:31.221756 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e504b99be3a84305a6c55fac00bf0086', 'request_id': 'req-52af6d76-e44d-4264-9466-dbac6c299a71'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:31.225675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e504b99be3a84305a6c55fac00bf0086', 'request_id': 'req-52af6d76-e44d-4264-9466-dbac6c299a71'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:31.233071 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e504b99be3a84305a6c55fac00bf0086', 'request_id': 'req-52af6d76-e44d-4264-9466-dbac6c299a71'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:31.310596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] The token cache is being invalidate because project e504b99be3a84305a6c55fac00bf0086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:31.310596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:31.310828 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:31.312552 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1302/2609] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:31 2022] DELETE /identity/v3/projects/e504b99be3a84305a6c55fac00bf0086 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:31.316842 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185903d3-8aa2-4715-b517-441f5b86d253 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:31.326555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185903d3-8aa2-4715-b517-441f5b86d253 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:31.391953 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:31.392619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:31.392903 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:31.393211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] PATH_INFO: `/v3/users/aa0347a1ac8b4c49a647f064f9d901f6` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:31.394356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0347a1ac8b4c49a647f064f9d901f6)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:31.396260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:31.637816 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0347a1ac8b4c49a647f064f9d901f6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:31.645492 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0347a1ac8b4c49a647f064f9d901f6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:31.715079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0347a1ac8b4c49a647f064f9d901f6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:31.715723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0347a1ac8b4c49a647f064f9d901f6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:31.730273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0347a1ac8b4c49a647f064f9d901f6'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:31.736710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1308/2610] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:31 2022] DELETE /identity/v3/users/aa0347a1ac8b4c49a647f064f9d901f6 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:32.162292 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:32.163728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:32.178159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af6d76-e44d-4264-9466-dbac6c299a71 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:32.254331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:32.254974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:32.255305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:32.255586 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] PATH_INFO: `/v3/projects/6133c4b9d21f4527b92b068d451ad272` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:32.256763 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6133c4b9d21f4527b92b068d451ad272)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:32.258693 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:32.387677 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:32.389694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:32.395316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6133c4b9d21f4527b92b068d451ad272', 'request_id': 'req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:32.400955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6133c4b9d21f4527b92b068d451ad272', 'request_id': 'req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:32.402891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607dbe7-7321-424b-81ae-402dc3038fee admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:32.407378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6133c4b9d21f4527b92b068d451ad272', 'request_id': 'req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:32.415235 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:32.459156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36c901-7598-4b5f-91af-38d0c85985f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:32.459779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c901-7598-4b5f-91af-38d0c85985f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:32.460058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c901-7598-4b5f-91af-38d0c85985f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:32.460329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36c901-7598-4b5f-91af-38d0c85985f2 admin admin] PATH_INFO: `/v3/users/ef4debe7f7f3494ab69c761516dd2980` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:32.461478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36c901-7598-4b5f-91af-38d0c85985f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef4debe7f7f3494ab69c761516dd2980)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:32.463606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36c901-7598-4b5f-91af-38d0c85985f2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:32.475625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] The token cache is being invalidate because project 6133c4b9d21f4527b92b068d451ad272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:32.476029 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:32.476702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b99ab4b4-381d-4cee-ad73-2a4a6fad0316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:32.478254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1303/2611] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:32 2022] DELETE /identity/v3/projects/6133c4b9d21f4527b92b068d451ad272 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:32.483601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515627a-2555-4f81-87f7-36ff4bf9cd53 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:32.483893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515627a-2555-4f81-87f7-36ff4bf9cd53 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:32.484172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515627a-2555-4f81-87f7-36ff4bf9cd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:32.503933 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9515627a-2555-4f81-87f7-36ff4bf9cd53 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:32.505167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9515627a-2555-4f81-87f7-36ff4bf9cd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:32.513178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1304/2612] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:32.520825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b67cb-e7a6-4477-bad2-3bac9891c04c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:32.521141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b67cb-e7a6-4477-bad2-3bac9891c04c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:32.521416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b67cb-e7a6-4477-bad2-3bac9891c04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:32.540504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9e8b67cb-e7a6-4477-bad2-3bac9891c04c None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:32.541774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9e8b67cb-e7a6-4477-bad2-3bac9891c04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:32.550249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1305/2613] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:07:32.554219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b67cb-e7a6-4477-bad2-3bac9891c04c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:32.563036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b67cb-e7a6-4477-bad2-3bac9891c04c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:32.567745 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022d67-fa90-4ad2-85c8-c5c7dd4eec09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:32.568309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022d67-fa90-4ad2-85c8-c5c7dd4eec09 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:32.568689 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022d67-fa90-4ad2-85c8-c5c7dd4eec09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:32.568969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f022d67-fa90-4ad2-85c8-c5c7dd4eec09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:32.569629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f022d67-fa90-4ad2-85c8-c5c7dd4eec09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:07:32.569934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f022d67-fa90-4ad2-85c8-c5c7dd4eec09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:32.571626 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f022d67-fa90-4ad2-85c8-c5c7dd4eec09 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:32.583634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1306/2614] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:07:32.587853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022d67-fa90-4ad2-85c8-c5c7dd4eec09 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:32.598353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f022d67-fa90-4ad2-85c8-c5c7dd4eec09 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:32.603102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78abcf-117e-4f7c-9619-394e27ef9811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:32.603674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78abcf-117e-4f7c-9619-394e27ef9811 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:32.603952 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78abcf-117e-4f7c-9619-394e27ef9811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:32.604229 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78abcf-117e-4f7c-9619-394e27ef9811 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:32.604927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78abcf-117e-4f7c-9619-394e27ef9811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:32.606859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78abcf-117e-4f7c-9619-394e27ef9811 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:32.655572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad36c901-7598-4b5f-91af-38d0c85985f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef4debe7f7f3494ab69c761516dd2980'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:32.662525 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad36c901-7598-4b5f-91af-38d0c85985f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef4debe7f7f3494ab69c761516dd2980'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:32.745660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1307/2615] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:32 2022] POST /identity/v3/projects => generated 401 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:07:32.749850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78abcf-117e-4f7c-9619-394e27ef9811 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:32.761309 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad36c901-7598-4b5f-91af-38d0c85985f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef4debe7f7f3494ab69c761516dd2980'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:32.768135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad36c901-7598-4b5f-91af-38d0c85985f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef4debe7f7f3494ab69c761516dd2980'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:32.768283 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:32.775092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad36c901-7598-4b5f-91af-38d0c85985f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef4debe7f7f3494ab69c761516dd2980'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:32.781959 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1309/2616] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:32 2022] DELETE /identity/v3/users/ef4debe7f7f3494ab69c761516dd2980 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:32.785987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a54a-eb53-481c-8268-56315386b6fc None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:32.786272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a54a-eb53-481c-8268-56315386b6fc None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:32.786544 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a54a-eb53-481c-8268-56315386b6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:32.805855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2a01a54a-eb53-481c-8268-56315386b6fc None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:32.807066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2a01a54a-eb53-481c-8268-56315386b6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:32.813456 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78abcf-117e-4f7c-9619-394e27ef9811 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:32.866423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1310/2617] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:07:32.876185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614fb6d-26cb-4edb-aa51-24cbda8d3bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:32.876481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01a54a-eb53-481c-8268-56315386b6fc None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:32.876890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614fb6d-26cb-4edb-aa51-24cbda8d3bea admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:32.877210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614fb6d-26cb-4edb-aa51-24cbda8d3bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:32.877488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614fb6d-26cb-4edb-aa51-24cbda8d3bea admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:32.877687 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2a01a54a-eb53-481c-8268-56315386b6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:32.878118 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614fb6d-26cb-4edb-aa51-24cbda8d3bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:32.880109 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614fb6d-26cb-4edb-aa51-24cbda8d3bea admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:32.920936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:32.936734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01a54a-eb53-481c-8268-56315386b6fc None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:32.940820 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cb53a-d481-459d-88e6-f9031718d840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:32.941644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb53a-d481-459d-88e6-f9031718d840 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:32.941935 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb53a-d481-459d-88e6-f9031718d840 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:32.942213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb53a-d481-459d-88e6-f9031718d840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:32.947299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1308/2618] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:32 2022] POST /identity/v3/users => generated 301 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:07:32.948357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614fb6d-26cb-4edb-aa51-24cbda8d3bea admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:32.949972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2614fb6d-26cb-4edb-aa51-24cbda8d3bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:32.952369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cb53a-d481-459d-88e6-f9031718d840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:32.953939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cb53a-d481-459d-88e6-f9031718d840 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:32.962600 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614fb6d-26cb-4edb-aa51-24cbda8d3bea admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:32.963983 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1311/2619] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:07:32.968571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06330ef7-f032-45f1-918a-66ef886bfddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:32.969202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06330ef7-f032-45f1-918a-66ef886bfddb admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:32.969508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06330ef7-f032-45f1-918a-66ef886bfddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:32.969943 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06330ef7-f032-45f1-918a-66ef886bfddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:32.970592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06330ef7-f032-45f1-918a-66ef886bfddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:32.972631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06330ef7-f032-45f1-918a-66ef886bfddb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:32.983312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1309/2620] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:32.988592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cb53a-d481-459d-88e6-f9031718d840 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:32.999185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cb53a-d481-459d-88e6-f9031718d840 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:33.004258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f1945-4afa-4d81-ab50-3b568a63167d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:33.004860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f1945-4afa-4d81-ab50-3b568a63167d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.005177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f1945-4afa-4d81-ab50-3b568a63167d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.005457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f1945-4afa-4d81-ab50-3b568a63167d admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:33.006103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f1945-4afa-4d81-ab50-3b568a63167d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:33.007710 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f1945-4afa-4d81-ab50-3b568a63167d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:33.019662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1312/2621] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:33.044342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06330ef7-f032-45f1-918a-66ef886bfddb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:33.055142 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06330ef7-f032-45f1-918a-66ef886bfddb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:33.059783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0466dd0-f6b4-438a-92f0-a0236999ba71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:33.060556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0466dd0-f6b4-438a-92f0-a0236999ba71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.060843 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0466dd0-f6b4-438a-92f0-a0236999ba71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.061141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0466dd0-f6b4-438a-92f0-a0236999ba71 admin admin] PATH_INFO: `/v3/projects/ba83bb06dfeb43c2a42ac5dea27c938b/users/25f5f50275664a769f36a297fcccb7ed/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:33.076800 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0466dd0-f6b4-438a-92f0-a0236999ba71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba83bb06dfeb43c2a42ac5dea27c938b, user_id=25f5f50275664a769f36a297fcccb7ed, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:33.079482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0466dd0-f6b4-438a-92f0-a0236999ba71 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:33.126958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1310/2622] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:33 2022] PUT /identity/v3/projects/ba83bb06dfeb43c2a42ac5dea27c938b/users/25f5f50275664a769f36a297fcccb7ed/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:33.137073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036b3d4-3bfc-4c3b-b360-7b748ea6526c None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.137645 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036b3d4-3bfc-4c3b-b360-7b748ea6526c None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.137995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f036b3d4-3bfc-4c3b-b360-7b748ea6526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:33.170423 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f036b3d4-3bfc-4c3b-b360-7b748ea6526c None None] MFA Rules not processed for user `25f5f50275664a769f36a297fcccb7ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:33.171905 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f036b3d4-3bfc-4c3b-b360-7b748ea6526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:33.245572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1313/2623] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:33 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:33.250736 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0466dd0-f6b4-438a-92f0-a0236999ba71 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:33.319562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0466dd0-f6b4-438a-92f0-a0236999ba71 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:33.324246 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022002f3-deed-493c-b849-d721171e2f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:33.324843 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022002f3-deed-493c-b849-d721171e2f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.325157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022002f3-deed-493c-b849-d721171e2f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.325435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022002f3-deed-493c-b849-d721171e2f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:33.326700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022002f3-deed-493c-b849-d721171e2f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:33.328599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022002f3-deed-493c-b849-d721171e2f6c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:33.339803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036b3d4-3bfc-4c3b-b360-7b748ea6526c None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:33.341724 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f036b3d4-3bfc-4c3b-b360-7b748ea6526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:33.356342 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f036b3d4-3bfc-4c3b-b360-7b748ea6526c None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:33.361122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:33.361759 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.362037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.362305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] PATH_INFO: `/v3/projects/ad093a10d59d4bbdacb896101cdb62d3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:33.363481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad093a10d59d4bbdacb896101cdb62d3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:33.365403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:33.402813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1311/2624] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:33 2022] POST /identity/v3/projects => generated 401 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:33.407251 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022002f3-deed-493c-b849-d721171e2f6c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:33.411505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:33.453122 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad093a10d59d4bbdacb896101cdb62d3', 'request_id': 'req-9e737a80-de28-4fde-b746-4a88839b9cbb'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:33.459403 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad093a10d59d4bbdacb896101cdb62d3', 'request_id': 'req-9e737a80-de28-4fde-b746-4a88839b9cbb'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:33.465929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad093a10d59d4bbdacb896101cdb62d3', 'request_id': 'req-9e737a80-de28-4fde-b746-4a88839b9cbb'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:33.473284 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022002f3-deed-493c-b849-d721171e2f6c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:33.530793 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad395dc1-4509-4611-b300-580dd50e2527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:33.531406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395dc1-4509-4611-b300-580dd50e2527 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.531686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395dc1-4509-4611-b300-580dd50e2527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.531959 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad395dc1-4509-4611-b300-580dd50e2527 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:33.532578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad395dc1-4509-4611-b300-580dd50e2527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:33.534502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad395dc1-4509-4611-b300-580dd50e2527 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:33.549944 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] The token cache is being invalidate because project ad093a10d59d4bbdacb896101cdb62d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:33.550263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:33.550964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:33.552587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1314/2625] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:33 2022] DELETE /identity/v3/projects/ad093a10d59d4bbdacb896101cdb62d3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:33.556446 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:33.557919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:33.570723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e737a80-de28-4fde-b746-4a88839b9cbb admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:33.575434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca884fd-3007-44d9-bfa2-c21d0f109510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:33.580253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca884fd-3007-44d9-bfa2-c21d0f109510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.580253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca884fd-3007-44d9-bfa2-c21d0f109510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.580253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca884fd-3007-44d9-bfa2-c21d0f109510 admin admin] PATH_INFO: `/v3/users/e8dd6704ce5f4a6ca8a8a2847b8f176d` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:33.580253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca884fd-3007-44d9-bfa2-c21d0f109510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8dd6704ce5f4a6ca8a8a2847b8f176d)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:33.580253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca884fd-3007-44d9-bfa2-c21d0f109510 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:33.629140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1312/2626] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:33 2022] POST /identity/v3/users => generated 307 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:07:33.632771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad395dc1-4509-4611-b300-580dd50e2527 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:33.634417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ad395dc1-4509-4611-b300-580dd50e2527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:33.646945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad395dc1-4509-4611-b300-580dd50e2527 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:33.651733 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4ec34-ccd8-469c-877f-8a4f01a04273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:33.658532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4ec34-ccd8-469c-877f-8a4f01a04273 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.658532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4ec34-ccd8-469c-877f-8a4f01a04273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.658532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4ec34-ccd8-469c-877f-8a4f01a04273 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:33.658532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4ec34-ccd8-469c-877f-8a4f01a04273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:33.658532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4ec34-ccd8-469c-877f-8a4f01a04273 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:33.665690 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1313/2627] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:07:33.669658 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4ec34-ccd8-469c-877f-8a4f01a04273 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:33.679338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4ec34-ccd8-469c-877f-8a4f01a04273 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:33.683922 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07fb24-a973-48fe-9e1b-fe280233812e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:33.687242 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07fb24-a973-48fe-9e1b-fe280233812e admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.687242 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07fb24-a973-48fe-9e1b-fe280233812e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.687242 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07fb24-a973-48fe-9e1b-fe280233812e admin admin] PATH_INFO: `/v3/projects/55cba06a070e4ac0a63a588f99f0a07e/users/f474d2b55eae469aa91ada99cad485da/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:33.699899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07fb24-a973-48fe-9e1b-fe280233812e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55cba06a070e4ac0a63a588f99f0a07e, user_id=f474d2b55eae469aa91ada99cad485da, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:33.703210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07fb24-a973-48fe-9e1b-fe280233812e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:33.772436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:33.781577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9ca884fd-3007-44d9-bfa2-c21d0f109510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8dd6704ce5f4a6ca8a8a2847b8f176d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:33.784461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1314/2628] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:33 2022] PUT /identity/v3/projects/55cba06a070e4ac0a63a588f99f0a07e/users/f474d2b55eae469aa91ada99cad485da/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:33.788353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07fb24-a973-48fe-9e1b-fe280233812e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:33.788795 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9ca884fd-3007-44d9-bfa2-c21d0f109510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8dd6704ce5f4a6ca8a8a2847b8f176d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:33.851449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07fb24-a973-48fe-9e1b-fe280233812e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:33.856228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2259193-a322-4300-bf40-45f14402fddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:33.856834 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259193-a322-4300-bf40-45f14402fddc admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.857156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259193-a322-4300-bf40-45f14402fddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.857576 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259193-a322-4300-bf40-45f14402fddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:33.858228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2259193-a322-4300-bf40-45f14402fddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:33.859802 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2259193-a322-4300-bf40-45f14402fddc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:33.863682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9ca884fd-3007-44d9-bfa2-c21d0f109510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8dd6704ce5f4a6ca8a8a2847b8f176d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:33.864400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9ca884fd-3007-44d9-bfa2-c21d0f109510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8dd6704ce5f4a6ca8a8a2847b8f176d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:33.865000 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:33.870345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1315/2629] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:07:33.874265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2259193-a322-4300-bf40-45f14402fddc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:33.876034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d2259193-a322-4300-bf40-45f14402fddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:33.881284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-9ca884fd-3007-44d9-bfa2-c21d0f109510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8dd6704ce5f4a6ca8a8a2847b8f176d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:33.887570 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1315/2630] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:33 2022] DELETE /identity/v3/users/e8dd6704ce5f4a6ca8a8a2847b8f176d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:33.889909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2259193-a322-4300-bf40-45f14402fddc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:33.949933 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283c35f-af5d-4faf-94de-40bdb9d0fd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:33.950594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283c35f-af5d-4faf-94de-40bdb9d0fd79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.950874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283c35f-af5d-4faf-94de-40bdb9d0fd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.951779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283c35f-af5d-4faf-94de-40bdb9d0fd79 admin admin] PATH_INFO: `/v3/projects/55cba06a070e4ac0a63a588f99f0a07e/users/f474d2b55eae469aa91ada99cad485da/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:33.954026 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283c35f-af5d-4faf-94de-40bdb9d0fd79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55cba06a070e4ac0a63a588f99f0a07e, user_id=f474d2b55eae469aa91ada99cad485da, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:33.956569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283c35f-af5d-4faf-94de-40bdb9d0fd79 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:33.967203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1316/2631] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:33 2022] PUT /identity/v3/projects/55cba06a070e4ac0a63a588f99f0a07e/users/f474d2b55eae469aa91ada99cad485da/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:33.978043 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652d68-a73b-44f6-946a-42593316d2ce None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:33.978323 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652d68-a73b-44f6-946a-42593316d2ce None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:33.978699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e652d68-a73b-44f6-946a-42593316d2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:34.007179 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-9e652d68-a73b-44f6-946a-42593316d2ce None None] MFA Rules not processed for user `f474d2b55eae469aa91ada99cad485da`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:34.008465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9e652d68-a73b-44f6-946a-42593316d2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:34.077462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1316/2632] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:33 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:34.088524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283c35f-af5d-4faf-94de-40bdb9d0fd79 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:34.090478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e283c35f-af5d-4faf-94de-40bdb9d0fd79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:34.153675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283c35f-af5d-4faf-94de-40bdb9d0fd79 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:34.158168 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fd3ac-8bab-4b89-8451-65297d418d0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:34.158779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fd3ac-8bab-4b89-8451-65297d418d0b service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:34.159410 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fd3ac-8bab-4b89-8451-65297d418d0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:34.159410 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fd3ac-8bab-4b89-8451-65297d418d0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:34.168733 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fd3ac-8bab-4b89-8451-65297d418d0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:34.170863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fd3ac-8bab-4b89-8451-65297d418d0b service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:34.180984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1317/2633] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:34 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:34.282602 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652d68-a73b-44f6-946a-42593316d2ce None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:34.284148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9e652d68-a73b-44f6-946a-42593316d2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:34.352711 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e652d68-a73b-44f6-946a-42593316d2ce None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:34.366598 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:34.367323 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:34.367606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:34.367885 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] PATH_INFO: `/v3/projects/202dbaf7927441a29261ba4fd2741c66` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:34.369080 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=202dbaf7927441a29261ba4fd2741c66)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:34.370988 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:34.398471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fd3ac-8bab-4b89-8451-65297d418d0b service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:34.407480 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fd3ac-8bab-4b89-8451-65297d418d0b service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:34.411121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdc3fb-68e3-43bc-8e18-16d9aa85763a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:34.416531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdc3fb-68e3-43bc-8e18-16d9aa85763a service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:34.416531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdc3fb-68e3-43bc-8e18-16d9aa85763a service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:34.416531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdc3fb-68e3-43bc-8e18-16d9aa85763a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:34.416531 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7efdc3fb-68e3-43bc-8e18-16d9aa85763a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:34.475118 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdc3fb-68e3-43bc-8e18-16d9aa85763a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:34.476642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdc3fb-68e3-43bc-8e18-16d9aa85763a service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:34.486694 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1318/2634] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:34 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:07:34.488524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:34.494936 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '202dbaf7927441a29261ba4fd2741c66', 'request_id': 'req-07c0fb8e-359b-4b87-86b7-7cd024e96210'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:34.502006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '202dbaf7927441a29261ba4fd2741c66', 'request_id': 'req-07c0fb8e-359b-4b87-86b7-7cd024e96210'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:34.508142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '202dbaf7927441a29261ba4fd2741c66', 'request_id': 'req-07c0fb8e-359b-4b87-86b7-7cd024e96210'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:34.567057 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] The token cache is being invalidate because project 202dbaf7927441a29261ba4fd2741c66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:34.567748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:34.568147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-07c0fb8e-359b-4b87-86b7-7cd024e96210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:34.569755 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1317/2635] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:34 2022] DELETE /identity/v3/projects/202dbaf7927441a29261ba4fd2741c66 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:34.575027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046dd1-548f-45f7-81bf-03d51368b84f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:34.575320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046dd1-548f-45f7-81bf-03d51368b84f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:34.575587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb046dd1-548f-45f7-81bf-03d51368b84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:34.598119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bb046dd1-548f-45f7-81bf-03d51368b84f None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:34.599345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bb046dd1-548f-45f7-81bf-03d51368b84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:34.637615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:34.666585 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1319/2636] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:34.675625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6bc0e-82ff-4237-97da-acd0a6e8d119 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:34.675987 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6bc0e-82ff-4237-97da-acd0a6e8d119 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:34.676269 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6bc0e-82ff-4237-97da-acd0a6e8d119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:34.697615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-72d6bc0e-82ff-4237-97da-acd0a6e8d119 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:34.698866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-72d6bc0e-82ff-4237-97da-acd0a6e8d119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:34.707093 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1318/2637] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:34.711631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb046dd1-548f-45f7-81bf-03d51368b84f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:34.722243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb046dd1-548f-45f7-81bf-03d51368b84f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:34.726935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b8336-d8f6-4899-a649-976162d2d20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:34.727537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b8336-d8f6-4899-a649-976162d2d20d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:34.727817 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b8336-d8f6-4899-a649-976162d2d20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:34.728091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b8336-d8f6-4899-a649-976162d2d20d admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:34.728735 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b8336-d8f6-4899-a649-976162d2d20d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:07:34.729061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b8336-d8f6-4899-a649-976162d2d20d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:34.730732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b8336-d8f6-4899-a649-976162d2d20d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:34.743210 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1320/2638] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:07:34.747450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6bc0e-82ff-4237-97da-acd0a6e8d119 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:34.757036 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6bc0e-82ff-4237-97da-acd0a6e8d119 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:34.761664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ba2ed-34bb-4ebf-94c1-a2c9c694901e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:34.762261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ba2ed-34bb-4ebf-94c1-a2c9c694901e admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:34.762551 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ba2ed-34bb-4ebf-94c1-a2c9c694901e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:34.762827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ba2ed-34bb-4ebf-94c1-a2c9c694901e admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:34.763509 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ba2ed-34bb-4ebf-94c1-a2c9c694901e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:34.765827 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ba2ed-34bb-4ebf-94c1-a2c9c694901e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:34.868841 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1319/2639] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:34 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:34.870012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b8336-d8f6-4899-a649-976162d2d20d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:34.880522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b8336-d8f6-4899-a649-976162d2d20d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:34.938627 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa3ee2-e3d8-4390-a457-e0ff53bea852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:34.939444 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa3ee2-e3d8-4390-a457-e0ff53bea852 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:34.939850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa3ee2-e3d8-4390-a457-e0ff53bea852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:34.940255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa3ee2-e3d8-4390-a457-e0ff53bea852 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:34.941228 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa3ee2-e3d8-4390-a457-e0ff53bea852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:34.943167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa3ee2-e3d8-4390-a457-e0ff53bea852 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:35.064601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1321/2640] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:34 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:35.070676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ba2ed-34bb-4ebf-94c1-a2c9c694901e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:35.081601 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ba2ed-34bb-4ebf-94c1-a2c9c694901e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:35.086527 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f998da9-c35c-46ce-b91e-b2f4c4dd0d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:35.087184 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998da9-c35c-46ce-b91e-b2f4c4dd0d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:35.087473 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998da9-c35c-46ce-b91e-b2f4c4dd0d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:35.087781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f998da9-c35c-46ce-b91e-b2f4c4dd0d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:35.088382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f998da9-c35c-46ce-b91e-b2f4c4dd0d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:35.089991 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f998da9-c35c-46ce-b91e-b2f4c4dd0d1e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:35.101680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1320/2641] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:35.105642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa3ee2-e3d8-4390-a457-e0ff53bea852 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:35.116730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa3ee2-e3d8-4390-a457-e0ff53bea852 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:35.121532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eec4b6-e450-402a-a3c8-08327a7c85de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:35.122152 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eec4b6-e450-402a-a3c8-08327a7c85de admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:35.122419 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eec4b6-e450-402a-a3c8-08327a7c85de admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:35.122692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eec4b6-e450-402a-a3c8-08327a7c85de admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:35.123672 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eec4b6-e450-402a-a3c8-08327a7c85de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:35.125345 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eec4b6-e450-402a-a3c8-08327a7c85de admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:35.139833 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1322/2642] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:35.141161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f998da9-c35c-46ce-b91e-b2f4c4dd0d1e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:35.152014 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f998da9-c35c-46ce-b91e-b2f4c4dd0d1e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:35.157131 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f946ede-6bd0-48ac-bf69-fdd02b727a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:35.158263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946ede-6bd0-48ac-bf69-fdd02b727a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:35.158798 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946ede-6bd0-48ac-bf69-fdd02b727a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:35.159180 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f946ede-6bd0-48ac-bf69-fdd02b727a52 admin admin] PATH_INFO: `/v3/projects/16f65c0b947b45beb4315615fddec1ca/users/9bbc79944fc647849cbcf7429c2f68ad/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:35.175891 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f946ede-6bd0-48ac-bf69-fdd02b727a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f65c0b947b45beb4315615fddec1ca, user_id=9bbc79944fc647849cbcf7429c2f68ad, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:35.178639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f946ede-6bd0-48ac-bf69-fdd02b727a52 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:35.275408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1321/2643] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:35 2022] PUT /identity/v3/projects/16f65c0b947b45beb4315615fddec1ca/users/9bbc79944fc647849cbcf7429c2f68ad/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:35.287742 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52e92f-bd0c-4045-afcb-6b02d4aa3014 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:35.288024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52e92f-bd0c-4045-afcb-6b02d4aa3014 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:35.288326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52e92f-bd0c-4045-afcb-6b02d4aa3014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:35.315796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5f52e92f-bd0c-4045-afcb-6b02d4aa3014 None None] MFA Rules not processed for user `9bbc79944fc647849cbcf7429c2f68ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:35.316967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5f52e92f-bd0c-4045-afcb-6b02d4aa3014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:35.381080 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1323/2644] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:35.385586 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f946ede-6bd0-48ac-bf69-fdd02b727a52 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:35.450340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f946ede-6bd0-48ac-bf69-fdd02b727a52 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:35.455153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d49092-0917-4793-8948-cb3c0f083d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:35.455770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d49092-0917-4793-8948-cb3c0f083d27 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:35.456029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d49092-0917-4793-8948-cb3c0f083d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:35.456299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d49092-0917-4793-8948-cb3c0f083d27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:35.456943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d49092-0917-4793-8948-cb3c0f083d27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:35.458866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d49092-0917-4793-8948-cb3c0f083d27 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:35.555228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1322/2645] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:35 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:35.559380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52e92f-bd0c-4045-afcb-6b02d4aa3014 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:35.570188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52e92f-bd0c-4045-afcb-6b02d4aa3014 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:35.601896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:35.629818 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5961bd6-7600-427e-9f3f-1ac6e8c9a474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:35.630426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5961bd6-7600-427e-9f3f-1ac6e8c9a474 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:35.630707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5961bd6-7600-427e-9f3f-1ac6e8c9a474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:35.631149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5961bd6-7600-427e-9f3f-1ac6e8c9a474 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:35.631774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5961bd6-7600-427e-9f3f-1ac6e8c9a474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:35.633744 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5961bd6-7600-427e-9f3f-1ac6e8c9a474 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:35.739170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1324/2646] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:35 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:35.743183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d49092-0917-4793-8948-cb3c0f083d27 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:35.753969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d49092-0917-4793-8948-cb3c0f083d27 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:35.758650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45de07d-2508-4ab9-8f7b-dfe7ba0c8006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:35.759239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45de07d-2508-4ab9-8f7b-dfe7ba0c8006 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:35.759516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45de07d-2508-4ab9-8f7b-dfe7ba0c8006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:35.759783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45de07d-2508-4ab9-8f7b-dfe7ba0c8006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:35.760417 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45de07d-2508-4ab9-8f7b-dfe7ba0c8006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:35.762020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45de07d-2508-4ab9-8f7b-dfe7ba0c8006 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:35.773459 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1323/2647] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:35.777630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5961bd6-7600-427e-9f3f-1ac6e8c9a474 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:35.788246 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5961bd6-7600-427e-9f3f-1ac6e8c9a474 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:35.793083 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36362ed4-6c8e-4490-87fe-07b8fbb5ebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:35.793753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36362ed4-6c8e-4490-87fe-07b8fbb5ebfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:35.794016 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36362ed4-6c8e-4490-87fe-07b8fbb5ebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:35.794289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36362ed4-6c8e-4490-87fe-07b8fbb5ebfc admin admin] PATH_INFO: `/v3/projects/eb895dfa7bff49b3bf7cd5243ce73d87/users/ec01309da0e64f4d91a7fd5046ebaea2/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:35.809477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36362ed4-6c8e-4490-87fe-07b8fbb5ebfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb895dfa7bff49b3bf7cd5243ce73d87, user_id=ec01309da0e64f4d91a7fd5046ebaea2, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:35.812099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36362ed4-6c8e-4490-87fe-07b8fbb5ebfc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:35.896698 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1325/2648] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:35 2022] PUT /identity/v3/projects/eb895dfa7bff49b3bf7cd5243ce73d87/users/ec01309da0e64f4d91a7fd5046ebaea2/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:35.900689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45de07d-2508-4ab9-8f7b-dfe7ba0c8006 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:35.911352 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45de07d-2508-4ab9-8f7b-dfe7ba0c8006 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:35.948916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:35.969105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b8e33-a9ab-43d2-a749-3501e9ddac71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:35.969745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b8e33-a9ab-43d2-a749-3501e9ddac71 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:35.970027 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b8e33-a9ab-43d2-a749-3501e9ddac71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:35.970297 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b8e33-a9ab-43d2-a749-3501e9ddac71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:35.970973 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b8e33-a9ab-43d2-a749-3501e9ddac71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:35.972582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b8e33-a9ab-43d2-a749-3501e9ddac71 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:35.982731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1324/2649] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:35.990561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36362ed4-6c8e-4490-87fe-07b8fbb5ebfc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:36.001224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36362ed4-6c8e-4490-87fe-07b8fbb5ebfc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:36.005910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf8cfd-0a1b-4ab0-bc8a-13cc50cb3c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:36.006564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf8cfd-0a1b-4ab0-bc8a-13cc50cb3c81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:36.006845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf8cfd-0a1b-4ab0-bc8a-13cc50cb3c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:36.007120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf8cfd-0a1b-4ab0-bc8a-13cc50cb3c81 admin admin] PATH_INFO: `/v3/projects/eb895dfa7bff49b3bf7cd5243ce73d87/users/ec01309da0e64f4d91a7fd5046ebaea2/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:36.009038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf8cfd-0a1b-4ab0-bc8a-13cc50cb3c81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb895dfa7bff49b3bf7cd5243ce73d87, user_id=ec01309da0e64f4d91a7fd5046ebaea2, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:36.011657 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf8cfd-0a1b-4ab0-bc8a-13cc50cb3c81 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:36.022686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1326/2650] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:35 2022] PUT /identity/v3/projects/eb895dfa7bff49b3bf7cd5243ce73d87/users/ec01309da0e64f4d91a7fd5046ebaea2/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:36.033714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db1ec5-3a55-4f85-b664-8e5175b0e910 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:36.034006 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db1ec5-3a55-4f85-b664-8e5175b0e910 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:36.034283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db1ec5-3a55-4f85-b664-8e5175b0e910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:36.061375 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-54db1ec5-3a55-4f85-b664-8e5175b0e910 None None] MFA Rules not processed for user `ec01309da0e64f4d91a7fd5046ebaea2`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:36.062650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-54db1ec5-3a55-4f85-b664-8e5175b0e910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:36.131504 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1325/2651] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:36.144937 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf8cfd-0a1b-4ab0-bc8a-13cc50cb3c81 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:36.146714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d5bf8cfd-0a1b-4ab0-bc8a-13cc50cb3c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:36.208771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf8cfd-0a1b-4ab0-bc8a-13cc50cb3c81 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:36.212453 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020d886-71e2-436b-97bd-8221c7e461c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:36.213091 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020d886-71e2-436b-97bd-8221c7e461c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:36.213372 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020d886-71e2-436b-97bd-8221c7e461c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:36.213642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020d886-71e2-436b-97bd-8221c7e461c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:36.272179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020d886-71e2-436b-97bd-8221c7e461c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:36.273632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020d886-71e2-436b-97bd-8221c7e461c7 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:36.282872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1327/2652] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:38.140028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db1ec5-3a55-4f85-b664-8e5175b0e910 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:38.141487 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-54db1ec5-3a55-4f85-b664-8e5175b0e910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:38.238897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db1ec5-3a55-4f85-b664-8e5175b0e910 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:38.242488 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730687cd-7dac-43f8-90e5-9dfa2a91487f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:38.243088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730687cd-7dac-43f8-90e5-9dfa2a91487f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:38.243373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730687cd-7dac-43f8-90e5-9dfa2a91487f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:38.243642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730687cd-7dac-43f8-90e5-9dfa2a91487f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:38.245405 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-730687cd-7dac-43f8-90e5-9dfa2a91487f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:38.290749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:38.327306 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730687cd-7dac-43f8-90e5-9dfa2a91487f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:38.328774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730687cd-7dac-43f8-90e5-9dfa2a91487f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:38.339294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1326/2653] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:39.468880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020d886-71e2-436b-97bd-8221c7e461c7 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:39.470397 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3020d886-71e2-436b-97bd-8221c7e461c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:39.515027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:39.585821 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020d886-71e2-436b-97bd-8221c7e461c7 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:39.590724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:39.591358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:39.592200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:39.592532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] PATH_INFO: `/v3/users/25f5f50275664a769f36a297fcccb7ed` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:39.594079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25f5f50275664a769f36a297fcccb7ed)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:39.596060 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:39.840416 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25f5f50275664a769f36a297fcccb7ed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:39.841265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25f5f50275664a769f36a297fcccb7ed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:39.842078 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25f5f50275664a769f36a297fcccb7ed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:39.888261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25f5f50275664a769f36a297fcccb7ed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:39.906194 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25f5f50275664a769f36a297fcccb7ed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:39.920695 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1328/2654] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:39 2022] DELETE /identity/v3/users/25f5f50275664a769f36a297fcccb7ed => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:39.925183 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987dd7c-6afb-452f-a83c-e31aee402db6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:39.925619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987dd7c-6afb-452f-a83c-e31aee402db6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:39.925714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987dd7c-6afb-452f-a83c-e31aee402db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:39.957828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0987dd7c-6afb-452f-a83c-e31aee402db6 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:39.959077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0987dd7c-6afb-452f-a83c-e31aee402db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:40.033596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1327/2655] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:07:40.043650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:40.045677 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:40.106749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6607329-29bd-4397-a77b-53a8c6fce738 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:40.110575 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371f61d-887c-40f9-a68e-ff8de05d5017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:40.111192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371f61d-887c-40f9-a68e-ff8de05d5017 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:40.111473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371f61d-887c-40f9-a68e-ff8de05d5017 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:40.111746 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371f61d-887c-40f9-a68e-ff8de05d5017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:40.121679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371f61d-887c-40f9-a68e-ff8de05d5017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:40.123934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371f61d-887c-40f9-a68e-ff8de05d5017 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:40.135178 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1329/2656] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:40.738664 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987dd7c-6afb-452f-a83c-e31aee402db6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:40.740211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0987dd7c-6afb-452f-a83c-e31aee402db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:40.754529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987dd7c-6afb-452f-a83c-e31aee402db6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:40.761209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:40.763003 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:40.763506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:40.763943 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] PATH_INFO: `/v3/projects/ba83bb06dfeb43c2a42ac5dea27c938b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:40.767901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba83bb06dfeb43c2a42ac5dea27c938b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:40.769425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:40.923062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba83bb06dfeb43c2a42ac5dea27c938b', 'request_id': 'req-137cc346-b76a-469a-809b-1f8b32cdc0db'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:40.930177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba83bb06dfeb43c2a42ac5dea27c938b', 'request_id': 'req-137cc346-b76a-469a-809b-1f8b32cdc0db'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:40.938166 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba83bb06dfeb43c2a42ac5dea27c938b', 'request_id': 'req-137cc346-b76a-469a-809b-1f8b32cdc0db'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:41.041859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] The token cache is being invalidate because project ba83bb06dfeb43c2a42ac5dea27c938b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:41.044288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:41.044288 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:41.046077 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1328/2657] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:40 2022] DELETE /identity/v3/projects/ba83bb06dfeb43c2a42ac5dea27c938b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:41.050004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371f61d-887c-40f9-a68e-ff8de05d5017 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:41.062778 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371f61d-887c-40f9-a68e-ff8de05d5017 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:41.073356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:41.145327 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:41.146033 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:41.146267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:41.146691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] PATH_INFO: `/v3/users/f474d2b55eae469aa91ada99cad485da` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:41.147883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f474d2b55eae469aa91ada99cad485da)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:41.150004 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:41.351028 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f474d2b55eae469aa91ada99cad485da'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:41.351806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f474d2b55eae469aa91ada99cad485da'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:41.352493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f474d2b55eae469aa91ada99cad485da'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:41.428616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f474d2b55eae469aa91ada99cad485da'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:41.442025 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f474d2b55eae469aa91ada99cad485da'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:41.455962 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1330/2658] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:41 2022] DELETE /identity/v3/users/f474d2b55eae469aa91ada99cad485da => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:41.629750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:41.631167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:41.678173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:41.692240 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cc346-b76a-469a-809b-1f8b32cdc0db admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:41.748748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906563a-dc3c-440f-9d63-d459922ad780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:41.749505 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906563a-dc3c-440f-9d63-d459922ad780 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:41.749800 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906563a-dc3c-440f-9d63-d459922ad780 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:41.750113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906563a-dc3c-440f-9d63-d459922ad780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:41.751975 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a906563a-dc3c-440f-9d63-d459922ad780 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:41.841149 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906563a-dc3c-440f-9d63-d459922ad780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:41.843250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906563a-dc3c-440f-9d63-d459922ad780 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:41.854189 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1329/2659] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:42.037458 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:42.039041 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:42.127175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef861ce-552b-4d5c-aec2-0ae593f78453 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:42.134199 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:42.134782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:42.135062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:42.135336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] PATH_INFO: `/v3/projects/55cba06a070e4ac0a63a588f99f0a07e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:42.136801 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55cba06a070e4ac0a63a588f99f0a07e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:42.138671 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:42.267469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:42.276883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55cba06a070e4ac0a63a588f99f0a07e', 'request_id': 'req-1edd6a83-378b-46e3-b65a-bffd1aeec048'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:42.282921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55cba06a070e4ac0a63a588f99f0a07e', 'request_id': 'req-1edd6a83-378b-46e3-b65a-bffd1aeec048'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:42.289204 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55cba06a070e4ac0a63a588f99f0a07e', 'request_id': 'req-1edd6a83-378b-46e3-b65a-bffd1aeec048'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:42.394982 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] The token cache is being invalidate because project 55cba06a070e4ac0a63a588f99f0a07e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:42.395358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:42.396039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1edd6a83-378b-46e3-b65a-bffd1aeec048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:42.397695 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1331/2660] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:42 2022] DELETE /identity/v3/projects/55cba06a070e4ac0a63a588f99f0a07e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:42.402462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39987c1-5f95-48e6-bdf4-fdb887ddb3d3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:42.402922 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39987c1-5f95-48e6-bdf4-fdb887ddb3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:42.403025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39987c1-5f95-48e6-bdf4-fdb887ddb3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:42.460511 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-c39987c1-5f95-48e6-bdf4-fdb887ddb3d3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:42.461797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c39987c1-5f95-48e6-bdf4-fdb887ddb3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:42.537084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1330/2661] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:42.547015 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73c9e4-1000-464a-b21a-f252d5aa3f82 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:42.547364 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73c9e4-1000-464a-b21a-f252d5aa3f82 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:42.547652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73c9e4-1000-464a-b21a-f252d5aa3f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:42.572322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-aa73c9e4-1000-464a-b21a-f252d5aa3f82 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:42.572322 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-aa73c9e4-1000-464a-b21a-f252d5aa3f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:42.583726 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1332/2662] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:42.589299 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39987c1-5f95-48e6-bdf4-fdb887ddb3d3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:42.600454 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39987c1-5f95-48e6-bdf4-fdb887ddb3d3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:42.605823 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8329f8a-cf95-4520-acaa-53c4dece1bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:42.606604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8329f8a-cf95-4520-acaa-53c4dece1bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:42.606892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8329f8a-cf95-4520-acaa-53c4dece1bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:42.608161 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8329f8a-cf95-4520-acaa-53c4dece1bd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:42.608895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8329f8a-cf95-4520-acaa-53c4dece1bd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:07:42.609237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8329f8a-cf95-4520-acaa-53c4dece1bd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:42.610997 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8329f8a-cf95-4520-acaa-53c4dece1bd1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:42.623273 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1331/2663] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:07:42.627926 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73c9e4-1000-464a-b21a-f252d5aa3f82 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:42.637268 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73c9e4-1000-464a-b21a-f252d5aa3f82 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:42.642422 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a45725-6fdb-492a-bd63-281170281097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:42.643014 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a45725-6fdb-492a-bd63-281170281097 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:42.643290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a45725-6fdb-492a-bd63-281170281097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:42.643564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a45725-6fdb-492a-bd63-281170281097 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:42.644192 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a45725-6fdb-492a-bd63-281170281097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:42.646092 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a45725-6fdb-492a-bd63-281170281097 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:42.717910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1333/2664] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:42 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:42.722477 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8329f8a-cf95-4520-acaa-53c4dece1bd1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:42.732552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8329f8a-cf95-4520-acaa-53c4dece1bd1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:42.737994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=100 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:42.789932 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4f993-1053-4cc1-9de2-9ffea1280c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:42.790550 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4f993-1053-4cc1-9de2-9ffea1280c42 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:42.790839 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4f993-1053-4cc1-9de2-9ffea1280c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:42.791112 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4f993-1053-4cc1-9de2-9ffea1280c42 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:42.791837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4f993-1053-4cc1-9de2-9ffea1280c42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:42.793884 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4f993-1053-4cc1-9de2-9ffea1280c42 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:42.873558 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1332/2665] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:42 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:07:42.874805 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a45725-6fdb-492a-bd63-281170281097 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:42.885760 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a45725-6fdb-492a-bd63-281170281097 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:42.891287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d0938-0be2-4996-8299-7e418f0bf88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:42.891915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d0938-0be2-4996-8299-7e418f0bf88f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:42.892182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d0938-0be2-4996-8299-7e418f0bf88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:42.892460 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d0938-0be2-4996-8299-7e418f0bf88f admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:42.893140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d0938-0be2-4996-8299-7e418f0bf88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:42.894727 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d0938-0be2-4996-8299-7e418f0bf88f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:42.905722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1334/2666] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:42.909808 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4f993-1053-4cc1-9de2-9ffea1280c42 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:42.920616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4f993-1053-4cc1-9de2-9ffea1280c42 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:42.925537 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badbba1-dc52-43af-b9ce-d01f0ed87b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:42.926147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badbba1-dc52-43af-b9ce-d01f0ed87b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:42.926445 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badbba1-dc52-43af-b9ce-d01f0ed87b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:42.926717 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badbba1-dc52-43af-b9ce-d01f0ed87b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:42.927359 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badbba1-dc52-43af-b9ce-d01f0ed87b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:42.929189 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badbba1-dc52-43af-b9ce-d01f0ed87b58 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:42.940703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1333/2667] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:42.945177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d0938-0be2-4996-8299-7e418f0bf88f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:42.956208 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d0938-0be2-4996-8299-7e418f0bf88f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:42.961960 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72540d-54d8-4cee-8dd6-886b3b74bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:42.962634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72540d-54d8-4cee-8dd6-886b3b74bf9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:42.962919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72540d-54d8-4cee-8dd6-886b3b74bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:42.963203 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72540d-54d8-4cee-8dd6-886b3b74bf9a admin admin] PATH_INFO: `/v3/projects/8f786ac1eaf34321a3c9d5f65e1165fc/users/2ba24e37bdfc4c09abb4258f4abcf8d4/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:42.981289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72540d-54d8-4cee-8dd6-886b3b74bf9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f786ac1eaf34321a3c9d5f65e1165fc, user_id=2ba24e37bdfc4c09abb4258f4abcf8d4, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:42.983789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72540d-54d8-4cee-8dd6-886b3b74bf9a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:43.059328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1335/2668] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:42 2022] PUT /identity/v3/projects/8f786ac1eaf34321a3c9d5f65e1165fc/users/2ba24e37bdfc4c09abb4258f4abcf8d4/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:43.070515 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474f478-4ff5-42e3-9d52-186ee7273da1 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:43.070855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474f478-4ff5-42e3-9d52-186ee7273da1 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:43.071132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474f478-4ff5-42e3-9d52-186ee7273da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:43.099888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-f474f478-4ff5-42e3-9d52-186ee7273da1 None None] MFA Rules not processed for user `2ba24e37bdfc4c09abb4258f4abcf8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:43.101144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f474f478-4ff5-42e3-9d52-186ee7273da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:43.169221 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1334/2669] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:07:43.180233 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72540d-54d8-4cee-8dd6-886b3b74bf9a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:43.181981 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-be72540d-54d8-4cee-8dd6-886b3b74bf9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:43.247361 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72540d-54d8-4cee-8dd6-886b3b74bf9a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:43.251896 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435b5a9-6c4c-4e59-9a74-1909d6faed07 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd83448e4c6704f7c8771132bd67eaa1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:43.252504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435b5a9-6c4c-4e59-9a74-1909d6faed07 service glance] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:43.252815 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435b5a9-6c4c-4e59-9a74-1909d6faed07 service glance] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:43.253124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435b5a9-6c4c-4e59-9a74-1909d6faed07 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:43.261702 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435b5a9-6c4c-4e59-9a74-1909d6faed07 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:43.263287 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435b5a9-6c4c-4e59-9a74-1909d6faed07 service glance] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:43.272669 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1336/2670] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:47.861980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474f478-4ff5-42e3-9d52-186ee7273da1 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:47.863396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-f474f478-4ff5-42e3-9d52-186ee7273da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:47.934386 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474f478-4ff5-42e3-9d52-186ee7273da1 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:47.939572 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:47.940218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:47.940497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:47.940776 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] PATH_INFO: `/v3/users/9bbc79944fc647849cbcf7429c2f68ad` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:47.941968 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bbc79944fc647849cbcf7429c2f68ad)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:47.943877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:47.965735 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:48.144969 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bbc79944fc647849cbcf7429c2f68ad'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:48.151530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bbc79944fc647849cbcf7429c2f68ad'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:48.222778 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bbc79944fc647849cbcf7429c2f68ad'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:48.223387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bbc79944fc647849cbcf7429c2f68ad'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:48.235930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bbc79944fc647849cbcf7429c2f68ad'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:48.241878 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1335/2671] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:47 2022] DELETE /identity/v3/users/9bbc79944fc647849cbcf7429c2f68ad => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:48.246494 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281b732-9fb4-4f23-9455-209e07b10178 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:48.246946 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281b732-9fb4-4f23-9455-209e07b10178 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:48.247084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281b732-9fb4-4f23-9455-209e07b10178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:48.267335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d281b732-9fb4-4f23-9455-209e07b10178 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:48.268563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d281b732-9fb4-4f23-9455-209e07b10178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:48.330315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1337/2672] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:48.341187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:48.342597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:48.401976 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b5026-555f-460a-9104-cae91696c8b6 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:48.405543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad517f04-a511-49fa-b256-0d28b7adf381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:48.406178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad517f04-a511-49fa-b256-0d28b7adf381 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:48.406460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad517f04-a511-49fa-b256-0d28b7adf381 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:48.406735 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad517f04-a511-49fa-b256-0d28b7adf381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:48.415859 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad517f04-a511-49fa-b256-0d28b7adf381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:48.417303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad517f04-a511-49fa-b256-0d28b7adf381 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:48.427704 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1336/2673] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:48.837217 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281b732-9fb4-4f23-9455-209e07b10178 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:48.839056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d281b732-9fb4-4f23-9455-209e07b10178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:48.847329 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:48.852950 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281b732-9fb4-4f23-9455-209e07b10178 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:48.857976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:48.858623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:48.858899 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:48.859701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] PATH_INFO: `/v3/projects/16f65c0b947b45beb4315615fddec1ca` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:48.861200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f65c0b947b45beb4315615fddec1ca)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:48.863140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:48.973655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f65c0b947b45beb4315615fddec1ca', 'request_id': 'req-2d308247-102e-4995-aae8-3112e3a11b88'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:48.979179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f65c0b947b45beb4315615fddec1ca', 'request_id': 'req-2d308247-102e-4995-aae8-3112e3a11b88'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:48.987188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f65c0b947b45beb4315615fddec1ca', 'request_id': 'req-2d308247-102e-4995-aae8-3112e3a11b88'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:49.058105 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] The token cache is being invalidate because project 16f65c0b947b45beb4315615fddec1ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:49.058402 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:49.059133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:49.060857 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1338/2674] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:48 2022] DELETE /identity/v3/projects/16f65c0b947b45beb4315615fddec1ca => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:49.065521 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad517f04-a511-49fa-b256-0d28b7adf381 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:49.075483 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad517f04-a511-49fa-b256-0d28b7adf381 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:49.132307 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2659099-9a8a-436c-944e-e019da8b6c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:49.132930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659099-9a8a-436c-944e-e019da8b6c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:49.133241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659099-9a8a-436c-944e-e019da8b6c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:49.133518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2659099-9a8a-436c-944e-e019da8b6c68 admin admin] PATH_INFO: `/v3/users/ec01309da0e64f4d91a7fd5046ebaea2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:49.134642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2659099-9a8a-436c-944e-e019da8b6c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec01309da0e64f4d91a7fd5046ebaea2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:49.136549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2659099-9a8a-436c-944e-e019da8b6c68 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:49.308497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:49.313709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a2659099-9a8a-436c-944e-e019da8b6c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec01309da0e64f4d91a7fd5046ebaea2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:49.320411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a2659099-9a8a-436c-944e-e019da8b6c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec01309da0e64f4d91a7fd5046ebaea2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:49.407861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a2659099-9a8a-436c-944e-e019da8b6c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec01309da0e64f4d91a7fd5046ebaea2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:49.408646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a2659099-9a8a-436c-944e-e019da8b6c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec01309da0e64f4d91a7fd5046ebaea2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:49.422085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a2659099-9a8a-436c-944e-e019da8b6c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec01309da0e64f4d91a7fd5046ebaea2'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:49.428442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1337/2675] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:49 2022] DELETE /identity/v3/users/ec01309da0e64f4d91a7fd5046ebaea2 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:49.837516 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:49.838994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:49.852626 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d308247-102e-4995-aae8-3112e3a11b88 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:49.922234 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:49.923023 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:49.923352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:49.923644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] PATH_INFO: `/v3/projects/eb895dfa7bff49b3bf7cd5243ce73d87` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:49.924900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb895dfa7bff49b3bf7cd5243ce73d87)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:49.926861 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:50.198094 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb895dfa7bff49b3bf7cd5243ce73d87', 'request_id': 'req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:50.208620 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb895dfa7bff49b3bf7cd5243ce73d87', 'request_id': 'req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:50.214939 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb895dfa7bff49b3bf7cd5243ce73d87', 'request_id': 'req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:50.280336 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] The token cache is being invalidate because project eb895dfa7bff49b3bf7cd5243ce73d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:50.280653 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:50.281312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e49da00b-4c9c-4967-9a8d-c3e5d95275ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:50.282942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1339/2676] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:49 2022] DELETE /identity/v3/projects/eb895dfa7bff49b3bf7cd5243ce73d87 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:50.288810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb571d-1958-42c1-a7cf-8b86174d64ee None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:50.289326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb571d-1958-42c1-a7cf-8b86174d64ee None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:50.289435 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb571d-1958-42c1-a7cf-8b86174d64ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:50.434583 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-94eb571d-1958-42c1-a7cf-8b86174d64ee None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:50.435854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-94eb571d-1958-42c1-a7cf-8b86174d64ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:50.502648 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1338/2677] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:50.515891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3029c1-3049-4a26-83dd-1cb0a4ceebb3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:50.516177 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3029c1-3049-4a26-83dd-1cb0a4ceebb3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:50.516453 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3029c1-3049-4a26-83dd-1cb0a4ceebb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:50.536369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9f3029c1-3049-4a26-83dd-1cb0a4ceebb3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:50.537599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9f3029c1-3049-4a26-83dd-1cb0a4ceebb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:50.545334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1340/2678] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:50.550239 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb571d-1958-42c1-a7cf-8b86174d64ee None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:50.559642 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb571d-1958-42c1-a7cf-8b86174d64ee None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:50.564284 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060ad25-b64b-4c5d-ba97-0c0d03b0258c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:50.564933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060ad25-b64b-4c5d-ba97-0c0d03b0258c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:50.565234 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060ad25-b64b-4c5d-ba97-0c0d03b0258c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:50.565501 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060ad25-b64b-4c5d-ba97-0c0d03b0258c admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:50.566164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060ad25-b64b-4c5d-ba97-0c0d03b0258c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:07:50.566465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060ad25-b64b-4c5d-ba97-0c0d03b0258c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:50.568056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060ad25-b64b-4c5d-ba97-0c0d03b0258c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:50.580457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1339/2679] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:07:50.585054 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3029c1-3049-4a26-83dd-1cb0a4ceebb3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:50.594425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3029c1-3049-4a26-83dd-1cb0a4ceebb3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:50.599614 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894f9cb-5d62-4ac0-a34d-781fbc835e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:50.600207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894f9cb-5d62-4ac0-a34d-781fbc835e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:50.600484 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894f9cb-5d62-4ac0-a34d-781fbc835e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:50.600751 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894f9cb-5d62-4ac0-a34d-781fbc835e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:50.601411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894f9cb-5d62-4ac0-a34d-781fbc835e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:50.603280 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894f9cb-5d62-4ac0-a34d-781fbc835e5d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:50.608461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:50.700069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1341/2680] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:50 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:50.704372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060ad25-b64b-4c5d-ba97-0c0d03b0258c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:50.714439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060ad25-b64b-4c5d-ba97-0c0d03b0258c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:50.770517 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f1984-11f7-4de7-8c59-9adfb15a9976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:50.771113 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f1984-11f7-4de7-8c59-9adfb15a9976 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:50.771507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f1984-11f7-4de7-8c59-9adfb15a9976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:50.771781 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f1984-11f7-4de7-8c59-9adfb15a9976 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:50.772540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f1984-11f7-4de7-8c59-9adfb15a9976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:50.774913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f1984-11f7-4de7-8c59-9adfb15a9976 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:50.869962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1340/2681] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:50 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:50.874293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894f9cb-5d62-4ac0-a34d-781fbc835e5d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:50.885665 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894f9cb-5d62-4ac0-a34d-781fbc835e5d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:50.891139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aec64-03c8-455e-986b-372a23d58d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:50.891762 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aec64-03c8-455e-986b-372a23d58d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:50.892567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aec64-03c8-455e-986b-372a23d58d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:50.892844 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aec64-03c8-455e-986b-372a23d58d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:50.893573 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4aec64-03c8-455e-986b-372a23d58d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:50.895162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4aec64-03c8-455e-986b-372a23d58d1e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:50.905698 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1342/2682] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:07:50.910865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f1984-11f7-4de7-8c59-9adfb15a9976 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:50.922280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f1984-11f7-4de7-8c59-9adfb15a9976 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:50.927256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c8527-a69d-4072-b69b-80b93ee3811d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:50.927865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c8527-a69d-4072-b69b-80b93ee3811d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:50.928146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c8527-a69d-4072-b69b-80b93ee3811d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:50.928420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c8527-a69d-4072-b69b-80b93ee3811d admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:50.929103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c8527-a69d-4072-b69b-80b93ee3811d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:50.930696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c8527-a69d-4072-b69b-80b93ee3811d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:50.941280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1341/2683] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:50.945554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aec64-03c8-455e-986b-372a23d58d1e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:50.956188 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aec64-03c8-455e-986b-372a23d58d1e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:50.960730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05073c6c-6fbe-48ee-8e7a-829682c7cac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:50.961415 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05073c6c-6fbe-48ee-8e7a-829682c7cac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:50.961700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05073c6c-6fbe-48ee-8e7a-829682c7cac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:50.961979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05073c6c-6fbe-48ee-8e7a-829682c7cac4 admin admin] PATH_INFO: `/v3/projects/2b37a1a9c6f74a15bbeb3a0f04e69ed2/users/101b91e7cfe54c708069cca119fcb6f6/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:50.977230 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05073c6c-6fbe-48ee-8e7a-829682c7cac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b37a1a9c6f74a15bbeb3a0f04e69ed2, user_id=101b91e7cfe54c708069cca119fcb6f6, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:50.979845 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05073c6c-6fbe-48ee-8e7a-829682c7cac4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:51.053519 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1343/2684] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:50 2022] PUT /identity/v3/projects/2b37a1a9c6f74a15bbeb3a0f04e69ed2/users/101b91e7cfe54c708069cca119fcb6f6/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:51.122029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253874c8-ea46-4c52-adeb-7db3444dbed9 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:51.122313 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253874c8-ea46-4c52-adeb-7db3444dbed9 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:51.122587 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253874c8-ea46-4c52-adeb-7db3444dbed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:51.153242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-253874c8-ea46-4c52-adeb-7db3444dbed9 None None] MFA Rules not processed for user `101b91e7cfe54c708069cca119fcb6f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:51.154457 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-253874c8-ea46-4c52-adeb-7db3444dbed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:51.166471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:51.226092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1342/2685] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:51.230987 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05073c6c-6fbe-48ee-8e7a-829682c7cac4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:51.296378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05073c6c-6fbe-48ee-8e7a-829682c7cac4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:51.302887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca509e-a278-4632-bc7a-e466d32fab57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:51.303593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca509e-a278-4632-bc7a-e466d32fab57 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:51.303976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca509e-a278-4632-bc7a-e466d32fab57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:51.304316 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca509e-a278-4632-bc7a-e466d32fab57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:51.305036 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca509e-a278-4632-bc7a-e466d32fab57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:51.307001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca509e-a278-4632-bc7a-e466d32fab57 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:51.378754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1344/2686] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:51 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:51.382829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253874c8-ea46-4c52-adeb-7db3444dbed9 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:51.394852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253874c8-ea46-4c52-adeb-7db3444dbed9 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:51.451818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453577ed-dc4d-4026-ae0a-d06ff7c63292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:51.452415 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453577ed-dc4d-4026-ae0a-d06ff7c63292 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:51.452694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453577ed-dc4d-4026-ae0a-d06ff7c63292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:51.452963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453577ed-dc4d-4026-ae0a-d06ff7c63292 admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:51.453613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453577ed-dc4d-4026-ae0a-d06ff7c63292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:51.455998 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453577ed-dc4d-4026-ae0a-d06ff7c63292 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:51.543912 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1343/2687] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:51 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:51.553273 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca509e-a278-4632-bc7a-e466d32fab57 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:51.564424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca509e-a278-4632-bc7a-e466d32fab57 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:51.569698 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8a975-6be6-40ef-bcba-b1477a837afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:51.570317 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8a975-6be6-40ef-bcba-b1477a837afd admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:51.570655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8a975-6be6-40ef-bcba-b1477a837afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:51.570994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8a975-6be6-40ef-bcba-b1477a837afd admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:51.571664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8a975-6be6-40ef-bcba-b1477a837afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:51.573479 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8a975-6be6-40ef-bcba-b1477a837afd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:51.586934 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1345/2688] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:07:51.591044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453577ed-dc4d-4026-ae0a-d06ff7c63292 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:51.602046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453577ed-dc4d-4026-ae0a-d06ff7c63292 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:51.606732 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccda475-e15c-4962-9766-d3c3c184a56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:51.607377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda475-e15c-4962-9766-d3c3c184a56c admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:51.607657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda475-e15c-4962-9766-d3c3c184a56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:51.607932 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda475-e15c-4962-9766-d3c3c184a56c admin admin] PATH_INFO: `/v3/projects/c49e3730bfef4ca0a31452e7d79d6c37/users/001bfa3adc954eaf85c9f1d9a41551f1/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:51.622397 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccda475-e15c-4962-9766-d3c3c184a56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49e3730bfef4ca0a31452e7d79d6c37, user_id=001bfa3adc954eaf85c9f1d9a41551f1, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:51.625060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccda475-e15c-4962-9766-d3c3c184a56c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:51.680787 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1344/2689] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:51 2022] PUT /identity/v3/projects/c49e3730bfef4ca0a31452e7d79d6c37/users/001bfa3adc954eaf85c9f1d9a41551f1/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:51.684885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8a975-6be6-40ef-bcba-b1477a837afd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:51.695369 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8a975-6be6-40ef-bcba-b1477a837afd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:51.757306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93e780-096d-42fb-8a5e-1ed856f5dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:51.757860 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93e780-096d-42fb-8a5e-1ed856f5dbab admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:51.758144 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93e780-096d-42fb-8a5e-1ed856f5dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:51.758420 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93e780-096d-42fb-8a5e-1ed856f5dbab admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:51.759073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93e780-096d-42fb-8a5e-1ed856f5dbab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:51.760668 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93e780-096d-42fb-8a5e-1ed856f5dbab admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:51.770819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1346/2690] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:51.775074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccda475-e15c-4962-9766-d3c3c184a56c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:51.785937 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccda475-e15c-4962-9766-d3c3c184a56c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:51.790548 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53244ab9-3f67-4b7e-b978-439f1dd4ef0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:51.791187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53244ab9-3f67-4b7e-b978-439f1dd4ef0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:51.791468 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53244ab9-3f67-4b7e-b978-439f1dd4ef0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:51.791742 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53244ab9-3f67-4b7e-b978-439f1dd4ef0a admin admin] PATH_INFO: `/v3/projects/c49e3730bfef4ca0a31452e7d79d6c37/users/001bfa3adc954eaf85c9f1d9a41551f1/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:51.793678 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53244ab9-3f67-4b7e-b978-439f1dd4ef0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c49e3730bfef4ca0a31452e7d79d6c37, user_id=001bfa3adc954eaf85c9f1d9a41551f1, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:51.796278 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53244ab9-3f67-4b7e-b978-439f1dd4ef0a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:51.806615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1345/2691] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:51 2022] PUT /identity/v3/projects/c49e3730bfef4ca0a31452e7d79d6c37/users/001bfa3adc954eaf85c9f1d9a41551f1/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:51.817330 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323d910-685e-432c-a413-89baed151aae None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:51.817617 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323d910-685e-432c-a413-89baed151aae None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:51.817894 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323d910-685e-432c-a413-89baed151aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:51.847068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a323d910-685e-432c-a413-89baed151aae None None] MFA Rules not processed for user `001bfa3adc954eaf85c9f1d9a41551f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:51.848294 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a323d910-685e-432c-a413-89baed151aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:51.858679 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:51.918074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1347/2692] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:51.929574 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53244ab9-3f67-4b7e-b978-439f1dd4ef0a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:51.931285 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-53244ab9-3f67-4b7e-b978-439f1dd4ef0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:51.990019 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53244ab9-3f67-4b7e-b978-439f1dd4ef0a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:51.993731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f1cdc-f049-49c4-8bb4-70a7919db3ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:51.994396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f1cdc-f049-49c4-8bb4-70a7919db3ba service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:51.994680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f1cdc-f049-49c4-8bb4-70a7919db3ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:51.994948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f1cdc-f049-49c4-8bb4-70a7919db3ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:52.013432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:52.049916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f1cdc-f049-49c4-8bb4-70a7919db3ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:52.051340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f1cdc-f049-49c4-8bb4-70a7919db3ba service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:52.060022 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1346/2693] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:55.034031 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323d910-685e-432c-a413-89baed151aae None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:55.095451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323d910-685e-432c-a413-89baed151aae None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:55.099973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:55.100538 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:55.100814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:55.101124 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] PATH_INFO: `/v3/users/101b91e7cfe54c708069cca119fcb6f6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:55.102230 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=101b91e7cfe54c708069cca119fcb6f6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:55.104222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:55.294512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101b91e7cfe54c708069cca119fcb6f6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:55.295157 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101b91e7cfe54c708069cca119fcb6f6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:55.295759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101b91e7cfe54c708069cca119fcb6f6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:55.348933 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101b91e7cfe54c708069cca119fcb6f6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:55.362518 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101b91e7cfe54c708069cca119fcb6f6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:55.373193 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1348/2694] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:55 2022] DELETE /identity/v3/users/101b91e7cfe54c708069cca119fcb6f6 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:55.378695 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57f8d7-ee98-4309-a81d-d7d1479602d3 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:55.380178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57f8d7-ee98-4309-a81d-d7d1479602d3 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:55.380178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57f8d7-ee98-4309-a81d-d7d1479602d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:55.400845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2d57f8d7-ee98-4309-a81d-d7d1479602d3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:55.402846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2d57f8d7-ee98-4309-a81d-d7d1479602d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:55.461857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1347/2695] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:55.472475 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:55.473920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:55.509966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:55.557179 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4efa13-edaa-4756-a64f-4ea92dfb8ea2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:55.560938 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af71732-a490-45f8-b01f-49c4404b536c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:55.561568 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af71732-a490-45f8-b01f-49c4404b536c service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:55.561846 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af71732-a490-45f8-b01f-49c4404b536c service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:55.562114 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af71732-a490-45f8-b01f-49c4404b536c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:55.571677 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af71732-a490-45f8-b01f-49c4404b536c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:55.573108 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af71732-a490-45f8-b01f-49c4404b536c service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:55.583278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1349/2696] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:07:55.968774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57f8d7-ee98-4309-a81d-d7d1479602d3 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:55.970624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2d57f8d7-ee98-4309-a81d-d7d1479602d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:55.983596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57f8d7-ee98-4309-a81d-d7d1479602d3 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:55.989400 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:55.990062 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:55.990340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:55.990608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] PATH_INFO: `/v3/projects/2b37a1a9c6f74a15bbeb3a0f04e69ed2` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:55.991791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b37a1a9c6f74a15bbeb3a0f04e69ed2)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:55.993913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:56.103965 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b37a1a9c6f74a15bbeb3a0f04e69ed2', 'request_id': 'req-daf8de1e-dc9a-4f4f-b595-1185949d6206'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:56.109404 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b37a1a9c6f74a15bbeb3a0f04e69ed2', 'request_id': 'req-daf8de1e-dc9a-4f4f-b595-1185949d6206'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:56.116468 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b37a1a9c6f74a15bbeb3a0f04e69ed2', 'request_id': 'req-daf8de1e-dc9a-4f4f-b595-1185949d6206'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:56.183543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] The token cache is being invalidate because project 2b37a1a9c6f74a15bbeb3a0f04e69ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:56.183972 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:56.184612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:56.186238 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1348/2697] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:55 2022] DELETE /identity/v3/projects/2b37a1a9c6f74a15bbeb3a0f04e69ed2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:56.190405 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af71732-a490-45f8-b01f-49c4404b536c service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:56.200159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af71732-a490-45f8-b01f-49c4404b536c service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:56.256587 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31741475-d8f0-4c23-8a4e-037e4e2c56dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:56.257272 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31741475-d8f0-4c23-8a4e-037e4e2c56dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:56.257567 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31741475-d8f0-4c23-8a4e-037e4e2c56dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:56.257842 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31741475-d8f0-4c23-8a4e-037e4e2c56dd admin admin] PATH_INFO: `/v3/users/001bfa3adc954eaf85c9f1d9a41551f1` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:56.258979 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31741475-d8f0-4c23-8a4e-037e4e2c56dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=001bfa3adc954eaf85c9f1d9a41551f1)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:56.260895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31741475-d8f0-4c23-8a4e-037e4e2c56dd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:56.403068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-31741475-d8f0-4c23-8a4e-037e4e2c56dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001bfa3adc954eaf85c9f1d9a41551f1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:56.403730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-31741475-d8f0-4c23-8a4e-037e4e2c56dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001bfa3adc954eaf85c9f1d9a41551f1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:56.404365 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-31741475-d8f0-4c23-8a4e-037e4e2c56dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001bfa3adc954eaf85c9f1d9a41551f1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:56.451984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-31741475-d8f0-4c23-8a4e-037e4e2c56dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001bfa3adc954eaf85c9f1d9a41551f1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:56.465394 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-31741475-d8f0-4c23-8a4e-037e4e2c56dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001bfa3adc954eaf85c9f1d9a41551f1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:56.478724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1350/2698] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:07:56 2022] DELETE /identity/v3/users/001bfa3adc954eaf85c9f1d9a41551f1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:07:56.927088 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:56.928599 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:56.942779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8de1e-dc9a-4f4f-b595-1185949d6206 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:56.985980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:57.000702 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:57.001373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:57.001652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:57.004254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] PATH_INFO: `/v3/projects/c49e3730bfef4ca0a31452e7d79d6c37` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:57.004254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c49e3730bfef4ca0a31452e7d79d6c37)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:57.005506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:57.154073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c49e3730bfef4ca0a31452e7d79d6c37', 'request_id': 'req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:57.159419 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c49e3730bfef4ca0a31452e7d79d6c37', 'request_id': 'req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:57.165412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c49e3730bfef4ca0a31452e7d79d6c37', 'request_id': 'req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:57.223518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] The token cache is being invalidate because project c49e3730bfef4ca0a31452e7d79d6c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:07:57.223803 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:57.224424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b6bc3021-4cbb-4531-95e9-8c9b1775fa20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:07:57.225950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1349/2699] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:07:56 2022] DELETE /identity/v3/projects/c49e3730bfef4ca0a31452e7d79d6c37 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:57.230724 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b773e5-28d3-4ac9-b132-ab96a6208c0d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:57.231182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b773e5-28d3-4ac9-b132-ab96a6208c0d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:57.231278 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b773e5-28d3-4ac9-b132-ab96a6208c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:57.250795 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-19b773e5-28d3-4ac9-b132-ab96a6208c0d None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:57.251947 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-19b773e5-28d3-4ac9-b132-ab96a6208c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:57.275956 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:57.310877 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1351/2700] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:57.318436 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775662bf-2d34-44c2-857f-d724e390a94e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:57.318723 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775662bf-2d34-44c2-857f-d724e390a94e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:57.319074 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775662bf-2d34-44c2-857f-d724e390a94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:57.339053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-775662bf-2d34-44c2-857f-d724e390a94e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:57.340251 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-775662bf-2d34-44c2-857f-d724e390a94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:57.349063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1350/2701] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:57.353196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b773e5-28d3-4ac9-b132-ab96a6208c0d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:57.362804 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b773e5-28d3-4ac9-b132-ab96a6208c0d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:57.367396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e0f7b-1e65-467b-86c9-62a1852d0a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:57.368002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e0f7b-1e65-467b-86c9-62a1852d0a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:57.368289 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e0f7b-1e65-467b-86c9-62a1852d0a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:57.368565 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e0f7b-1e65-467b-86c9-62a1852d0a1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:57.369253 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e0f7b-1e65-467b-86c9-62a1852d0a1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:07:57.369561 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e0f7b-1e65-467b-86c9-62a1852d0a1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:57.371134 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e0f7b-1e65-467b-86c9-62a1852d0a1d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:57.382583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1352/2702] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:07:57.386712 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775662bf-2d34-44c2-857f-d724e390a94e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:57.396380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775662bf-2d34-44c2-857f-d724e390a94e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:57.400994 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbf7c6-b2b1-4f9a-856c-1ec39a7f5e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:57.401614 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbf7c6-b2b1-4f9a-856c-1ec39a7f5e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:57.401894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbf7c6-b2b1-4f9a-856c-1ec39a7f5e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:57.402167 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbf7c6-b2b1-4f9a-856c-1ec39a7f5e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:57.402784 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbf7c6-b2b1-4f9a-856c-1ec39a7f5e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:57.404657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbf7c6-b2b1-4f9a-856c-1ec39a7f5e29 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:57.488608 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1351/2703] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:07:57 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:57.492923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e0f7b-1e65-467b-86c9-62a1852d0a1d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:57.503996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e0f7b-1e65-467b-86c9-62a1852d0a1d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:57.562030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d05ab-b913-47cd-955a-b1d4120b010f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:57.562632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d05ab-b913-47cd-955a-b1d4120b010f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:57.562908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d05ab-b913-47cd-955a-b1d4120b010f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:57.563175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d05ab-b913-47cd-955a-b1d4120b010f admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:57.563789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d05ab-b913-47cd-955a-b1d4120b010f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:57.566530 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d05ab-b913-47cd-955a-b1d4120b010f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:57.669513 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1353/2704] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:07:57 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:07:57.673911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbf7c6-b2b1-4f9a-856c-1ec39a7f5e29 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:57.684363 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbf7c6-b2b1-4f9a-856c-1ec39a7f5e29 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:57.688982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e56f7-8c52-4281-920a-856de1a40a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:57.690090 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e56f7-8c52-4281-920a-856de1a40a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:57.690372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e56f7-8c52-4281-920a-856de1a40a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:57.690688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e56f7-8c52-4281-920a-856de1a40a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:57.691315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e56f7-8c52-4281-920a-856de1a40a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:57.692913 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e56f7-8c52-4281-920a-856de1a40a52 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:57.702582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1352/2705] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:07:57.707175 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d05ab-b913-47cd-955a-b1d4120b010f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:57.717098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d05ab-b913-47cd-955a-b1d4120b010f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:57.722137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c1ab0-c4d2-41f1-aced-36e80ff341c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:57.722765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c1ab0-c4d2-41f1-aced-36e80ff341c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:57.723044 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c1ab0-c4d2-41f1-aced-36e80ff341c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:57.723319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c1ab0-c4d2-41f1-aced-36e80ff341c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:57.723989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c1ab0-c4d2-41f1-aced-36e80ff341c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:57.725596 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c1ab0-c4d2-41f1-aced-36e80ff341c4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:57.735359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1354/2706] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:07:57.739350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e56f7-8c52-4281-920a-856de1a40a52 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:57.749107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e56f7-8c52-4281-920a-856de1a40a52 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:57.754266 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d72ac3-8fc6-4e01-946b-c05a3596ff95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:57.754835 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d72ac3-8fc6-4e01-946b-c05a3596ff95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:57.755110 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d72ac3-8fc6-4e01-946b-c05a3596ff95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:57.755377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d72ac3-8fc6-4e01-946b-c05a3596ff95 admin admin] PATH_INFO: `/v3/projects/a8690060da11478694b7110a10a00a21/users/89cc9499c67e41929b2289cfb7dd4a74/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:57.769888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d72ac3-8fc6-4e01-946b-c05a3596ff95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8690060da11478694b7110a10a00a21, user_id=89cc9499c67e41929b2289cfb7dd4a74, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:57.772470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d72ac3-8fc6-4e01-946b-c05a3596ff95 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:57.822622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1353/2707] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:07:57 2022] PUT /identity/v3/projects/a8690060da11478694b7110a10a00a21/users/89cc9499c67e41929b2289cfb7dd4a74/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:07:57.828383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741f681-289a-4904-9620-134b3a83cb93 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:57.828850 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741f681-289a-4904-9620-134b3a83cb93 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:57.829353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741f681-289a-4904-9620-134b3a83cb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:57.849876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3741f681-289a-4904-9620-134b3a83cb93 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:57.851073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3741f681-289a-4904-9620-134b3a83cb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:57.911404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1355/2708] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:57.921164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d72ac3-8fc6-4e01-946b-c05a3596ff95 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:57.922844 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b0d72ac3-8fc6-4e01-946b-c05a3596ff95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:57.979779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d72ac3-8fc6-4e01-946b-c05a3596ff95 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:57.983393 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14c205-defd-4e0b-a7db-d70f7207f8fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:57.983991 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14c205-defd-4e0b-a7db-d70f7207f8fc service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:57.984272 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14c205-defd-4e0b-a7db-d70f7207f8fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:57.984547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14c205-defd-4e0b-a7db-d70f7207f8fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:57.994058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14c205-defd-4e0b-a7db-d70f7207f8fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:57.995476 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14c205-defd-4e0b-a7db-d70f7207f8fc service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:58.005458 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1354/2709] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:07:59.174090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b680a9c-1e87-421d-9133-80e13b0d0a60 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:59.174563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b680a9c-1e87-421d-9133-80e13b0d0a60 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:59.174778 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b680a9c-1e87-421d-9133-80e13b0d0a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:59.205914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8b680a9c-1e87-421d-9133-80e13b0d0a60 None None] MFA Rules not processed for user `89cc9499c67e41929b2289cfb7dd4a74`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:07:59.207341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8b680a9c-1e87-421d-9133-80e13b0d0a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:59.231890 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:59.271821 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1356/2710] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:07:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:07:59.282815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14c205-defd-4e0b-a7db-d70f7207f8fc service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:07:59.284838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ff14c205-defd-4e0b-a7db-d70f7207f8fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:07:59.299739 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:07:59.340604 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14c205-defd-4e0b-a7db-d70f7207f8fc service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:07:59.344497 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de9ee6-6f6f-4d6d-81c7-6e2071d76e6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:07:59.345128 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de9ee6-6f6f-4d6d-81c7-6e2071d76e6a service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:07:59.345410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de9ee6-6f6f-4d6d-81c7-6e2071d76e6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:07:59.346303 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de9ee6-6f6f-4d6d-81c7-6e2071d76e6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:07:59.354646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de9ee6-6f6f-4d6d-81c7-6e2071d76e6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:07:59.356075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de9ee6-6f6f-4d6d-81c7-6e2071d76e6a service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:07:59.365640 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1355/2711] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:07:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:01.079148 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab4973-bcec-4bb1-8f30-aba3e05a1a1c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:01.079623 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab4973-bcec-4bb1-8f30-aba3e05a1a1c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:01.079738 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab4973-bcec-4bb1-8f30-aba3e05a1a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:01.100047 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a8ab4973-bcec-4bb1-8f30-aba3e05a1a1c None None] MFA Rules not processed for user `89cc9499c67e41929b2289cfb7dd4a74`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:01.101313 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a8ab4973-bcec-4bb1-8f30-aba3e05a1a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:01.108190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1357/2712] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:03.872442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de9ee6-6f6f-4d6d-81c7-6e2071d76e6a service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:03.874424 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b7de9ee6-6f6f-4d6d-81c7-6e2071d76e6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:03.888202 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de9ee6-6f6f-4d6d-81c7-6e2071d76e6a service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:03.892694 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:03.893882 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:03.894154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:03.894428 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] PATH_INFO: `/v3/users/2ba24e37bdfc4c09abb4258f4abcf8d4` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:03.895555 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ba24e37bdfc4c09abb4258f4abcf8d4)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:03.897492 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:04.100794 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ba24e37bdfc4c09abb4258f4abcf8d4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:04.107843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ba24e37bdfc4c09abb4258f4abcf8d4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:04.172552 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ba24e37bdfc4c09abb4258f4abcf8d4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:04.173226 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ba24e37bdfc4c09abb4258f4abcf8d4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:04.187329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ba24e37bdfc4c09abb4258f4abcf8d4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:04.197921 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1356/2713] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:08:03 2022] DELETE /identity/v3/users/2ba24e37bdfc4c09abb4258f4abcf8d4 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:04.199645 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51b6b1-fbba-4d02-a882-47f41e576253 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:04.200335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51b6b1-fbba-4d02-a882-47f41e576253 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:04.201370 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51b6b1-fbba-4d02-a882-47f41e576253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:04.223790 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cc51b6b1-fbba-4d02-a882-47f41e576253 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:04.225237 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cc51b6b1-fbba-4d02-a882-47f41e576253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:04.289975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1358/2714] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:04.299904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:04.301421 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:04.368870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758cb8f-a7ff-499f-bdf6-c9643946aca0 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:04.374254 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e30d3-2a42-451d-96e1-31561468ff56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:04.375490 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e30d3-2a42-451d-96e1-31561468ff56 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:04.375842 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e30d3-2a42-451d-96e1-31561468ff56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:04.376053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e30d3-2a42-451d-96e1-31561468ff56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:04.389559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e30d3-2a42-451d-96e1-31561468ff56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:04.391067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e30d3-2a42-451d-96e1-31561468ff56 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:04.403137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1357/2715] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:04.516725 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51b6b1-fbba-4d02-a882-47f41e576253 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:04.518858 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cc51b6b1-fbba-4d02-a882-47f41e576253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:04.609807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51b6b1-fbba-4d02-a882-47f41e576253 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:04.613594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da81e16-b6ea-4008-b71b-b70a8ee5bbcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:04.614211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da81e16-b6ea-4008-b71b-b70a8ee5bbcf service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:04.614491 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da81e16-b6ea-4008-b71b-b70a8ee5bbcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:04.614766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da81e16-b6ea-4008-b71b-b70a8ee5bbcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:04.616663 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0da81e16-b6ea-4008-b71b-b70a8ee5bbcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:04.688162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:04.693233 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da81e16-b6ea-4008-b71b-b70a8ee5bbcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:04.694693 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da81e16-b6ea-4008-b71b-b70a8ee5bbcf service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:04.706504 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1359/2716] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:04.842821 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e30d3-2a42-451d-96e1-31561468ff56 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:04.844669 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-979e30d3-2a42-451d-96e1-31561468ff56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:04.862622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e30d3-2a42-451d-96e1-31561468ff56 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:04.879371 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:04.880021 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:04.880327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:04.882620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] PATH_INFO: `/v3/projects/8f786ac1eaf34321a3c9d5f65e1165fc` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:04.882620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f786ac1eaf34321a3c9d5f65e1165fc)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:04.883690 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:05.007153 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:05.024734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f786ac1eaf34321a3c9d5f65e1165fc', 'request_id': 'req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:05.030324 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f786ac1eaf34321a3c9d5f65e1165fc', 'request_id': 'req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:05.039912 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f786ac1eaf34321a3c9d5f65e1165fc', 'request_id': 'req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:05.126295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] The token cache is being invalidate because project 8f786ac1eaf34321a3c9d5f65e1165fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:08:05.129590 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:05.129590 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-02f70fdc-d73b-4b68-bd97-c99d5a4b02e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:05.130098 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1358/2717] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:08:04 2022] DELETE /identity/v3/projects/8f786ac1eaf34321a3c9d5f65e1165fc => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:05.135748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc386b0-be01-4a25-b4ad-2b3b6404824b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:05.136035 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc386b0-be01-4a25-b4ad-2b3b6404824b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:05.136315 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc386b0-be01-4a25-b4ad-2b3b6404824b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:05.156378 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-dcc386b0-be01-4a25-b4ad-2b3b6404824b None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:05.164255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-dcc386b0-be01-4a25-b4ad-2b3b6404824b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:05.237426 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1360/2718] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:05.245453 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e517-f436-4d80-a3a8-3df4026f9930 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:05.245737 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e517-f436-4d80-a3a8-3df4026f9930 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:05.246008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7e517-f436-4d80-a3a8-3df4026f9930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:05.268344 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-34e7e517-f436-4d80-a3a8-3df4026f9930 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:05.269612 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-34e7e517-f436-4d80-a3a8-3df4026f9930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:05.279543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1359/2719] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:05.284841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc386b0-be01-4a25-b4ad-2b3b6404824b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:05.292763 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc386b0-be01-4a25-b4ad-2b3b6404824b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:05.297799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95ea35-9d95-44a6-b6f0-55d78b23efbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:05.298525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95ea35-9d95-44a6-b6f0-55d78b23efbb admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:05.298870 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95ea35-9d95-44a6-b6f0-55d78b23efbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:05.299207 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95ea35-9d95-44a6-b6f0-55d78b23efbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:05.300866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95ea35-9d95-44a6-b6f0-55d78b23efbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:08:05.301258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95ea35-9d95-44a6-b6f0-55d78b23efbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:05.302994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95ea35-9d95-44a6-b6f0-55d78b23efbb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:05.317128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1361/2720] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:08:05.322465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7e517-f436-4d80-a3a8-3df4026f9930 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:05.332085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7e517-f436-4d80-a3a8-3df4026f9930 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:05.336911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad2bee-0efc-4391-ab04-89a6acf4af59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:05.337624 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad2bee-0efc-4391-ab04-89a6acf4af59 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:05.337908 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad2bee-0efc-4391-ab04-89a6acf4af59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:05.338232 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad2bee-0efc-4391-ab04-89a6acf4af59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:05.338814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad2bee-0efc-4391-ab04-89a6acf4af59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:05.340692 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad2bee-0efc-4391-ab04-89a6acf4af59 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:05.477136 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1360/2721] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:08:05 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:08:05.480914 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95ea35-9d95-44a6-b6f0-55d78b23efbb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:05.495388 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95ea35-9d95-44a6-b6f0-55d78b23efbb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:05.546883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9cb0e-dfae-41a6-83fc-39081e0ef947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:05.547485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9cb0e-dfae-41a6-83fc-39081e0ef947 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:05.547782 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9cb0e-dfae-41a6-83fc-39081e0ef947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:05.548087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9cb0e-dfae-41a6-83fc-39081e0ef947 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:05.548710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9cb0e-dfae-41a6-83fc-39081e0ef947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:05.551357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9cb0e-dfae-41a6-83fc-39081e0ef947 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:05.646686 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1362/2722] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:08:05 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:05.652028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad2bee-0efc-4391-ab04-89a6acf4af59 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:05.663395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad2bee-0efc-4391-ab04-89a6acf4af59 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:05.668417 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871c753-e161-4bb0-97eb-01098a2780da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:05.668862 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871c753-e161-4bb0-97eb-01098a2780da admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:05.669176 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871c753-e161-4bb0-97eb-01098a2780da admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:05.669450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871c753-e161-4bb0-97eb-01098a2780da admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:05.670101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871c753-e161-4bb0-97eb-01098a2780da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:05.671730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871c753-e161-4bb0-97eb-01098a2780da admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:05.683486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1361/2723] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:08:05.687357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9cb0e-dfae-41a6-83fc-39081e0ef947 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:05.697542 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9cb0e-dfae-41a6-83fc-39081e0ef947 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:05.702233 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85ca0c-f4db-4bea-8d8b-ca07ef8f4737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:05.702863 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85ca0c-f4db-4bea-8d8b-ca07ef8f4737 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:05.703141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85ca0c-f4db-4bea-8d8b-ca07ef8f4737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:05.703408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85ca0c-f4db-4bea-8d8b-ca07ef8f4737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:05.704075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85ca0c-f4db-4bea-8d8b-ca07ef8f4737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:05.705951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85ca0c-f4db-4bea-8d8b-ca07ef8f4737 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:05.720643 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1363/2724] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:08:05.722494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871c753-e161-4bb0-97eb-01098a2780da admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:05.736259 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871c753-e161-4bb0-97eb-01098a2780da admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:05.737828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b5398-6ff1-403b-9b48-20b84f98be9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:05.738476 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b5398-6ff1-403b-9b48-20b84f98be9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:05.738749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b5398-6ff1-403b-9b48-20b84f98be9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:05.739017 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b5398-6ff1-403b-9b48-20b84f98be9c admin admin] PATH_INFO: `/v3/projects/5a8afe85664248fd83617aab4e45c442/users/f4a8f2f254eb4bf9bb8549ab66f46d22/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:05.754794 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b5398-6ff1-403b-9b48-20b84f98be9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8afe85664248fd83617aab4e45c442, user_id=f4a8f2f254eb4bf9bb8549ab66f46d22, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:05.757482 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b5398-6ff1-403b-9b48-20b84f98be9c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:05.852506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1362/2725] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:08:05 2022] PUT /identity/v3/projects/5a8afe85664248fd83617aab4e45c442/users/f4a8f2f254eb4bf9bb8549ab66f46d22/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:05.920073 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea231b-f22c-4848-932d-5040d71b0e20 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:05.936592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea231b-f22c-4848-932d-5040d71b0e20 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:05.936592 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea231b-f22c-4848-932d-5040d71b0e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:05.949618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4bea231b-f22c-4848-932d-5040d71b0e20 None None] MFA Rules not processed for user `f4a8f2f254eb4bf9bb8549ab66f46d22`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:05.950931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4bea231b-f22c-4848-932d-5040d71b0e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:05.997883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:06.015314 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1364/2726] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:06.026588 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b5398-6ff1-403b-9b48-20b84f98be9c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:06.028270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-c61b5398-6ff1-403b-9b48-20b84f98be9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:06.084466 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b5398-6ff1-403b-9b48-20b84f98be9c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:06.088094 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496f1a4-be3c-4922-8e8a-97041e3d28b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:06.088696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496f1a4-be3c-4922-8e8a-97041e3d28b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:06.088981 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496f1a4-be3c-4922-8e8a-97041e3d28b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:06.089285 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496f1a4-be3c-4922-8e8a-97041e3d28b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:06.098040 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496f1a4-be3c-4922-8e8a-97041e3d28b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:06.099460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496f1a4-be3c-4922-8e8a-97041e3d28b9 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:06.108105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1363/2727] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:08.926912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b63a-8d89-47a4-a716-a11fcc724505 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:08.927688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b63a-8d89-47a4-a716-a11fcc724505 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:08.927688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b63a-8d89-47a4-a716-a11fcc724505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:08.946593 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1bc5b63a-8d89-47a4-a716-a11fcc724505 None None] MFA Rules not processed for user `f4a8f2f254eb4bf9bb8549ab66f46d22`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:08.947796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1bc5b63a-8d89-47a4-a716-a11fcc724505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:08.954673 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1365/2728] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:12.373253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496f1a4-be3c-4922-8e8a-97041e3d28b9 service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:12.375420 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1496f1a4-be3c-4922-8e8a-97041e3d28b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:12.451390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:12.486275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496f1a4-be3c-4922-8e8a-97041e3d28b9 service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:12.493714 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282978b-37e6-4c6f-ae28-1c557d6f2a65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:12.494416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282978b-37e6-4c6f-ae28-1c557d6f2a65 service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:12.495315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282978b-37e6-4c6f-ae28-1c557d6f2a65 service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:12.495315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282978b-37e6-4c6f-ae28-1c557d6f2a65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:12.498487 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8282978b-37e6-4c6f-ae28-1c557d6f2a65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:12.517644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282978b-37e6-4c6f-ae28-1c557d6f2a65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:12.520066 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282978b-37e6-4c6f-ae28-1c557d6f2a65 service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:12.532156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1364/2729] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:13.339705 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38f2a8-51a9-4542-896d-dc1d1352f324 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:13.340235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38f2a8-51a9-4542-896d-dc1d1352f324 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:13.340338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c38f2a8-51a9-4542-896d-dc1d1352f324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:13.359973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3c38f2a8-51a9-4542-896d-dc1d1352f324 None None] MFA Rules not processed for user `ee81d8d9cf994df68a15afe5cb9673c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:13.361267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3c38f2a8-51a9-4542-896d-dc1d1352f324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:13.416525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1366/2730] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:20.704543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282978b-37e6-4c6f-ae28-1c557d6f2a65 service placement] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:20.706906 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8282978b-37e6-4c6f-ae28-1c557d6f2a65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:20.771260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282978b-37e6-4c6f-ae28-1c557d6f2a65 service placement] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:20.774661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ee404-f71b-442c-8128-9c8a38b95f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:20.775269 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ee404-f71b-442c-8128-9c8a38b95f0b service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:20.775549 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ee404-f71b-442c-8128-9c8a38b95f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:20.775846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ee404-f71b-442c-8128-9c8a38b95f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:20.777611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-217ee404-f71b-442c-8128-9c8a38b95f0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:20.852911 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ee404-f71b-442c-8128-9c8a38b95f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:20.854365 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ee404-f71b-442c-8128-9c8a38b95f0b service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:20.866372 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1365/2731] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:23.409297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=33 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:24.332358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7315bf8-537d-4938-9cc1-567f568f2e43 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:24.332874 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7315bf8-537d-4938-9cc1-567f568f2e43 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:24.333173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7315bf8-537d-4938-9cc1-567f568f2e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:24.353913 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a7315bf8-537d-4938-9cc1-567f568f2e43 None None] MFA Rules not processed for user `89cc9499c67e41929b2289cfb7dd4a74`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:24.355346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a7315bf8-537d-4938-9cc1-567f568f2e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:24.421176 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1367/2732] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:24.929429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ee404-f71b-442c-8128-9c8a38b95f0b service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:24.931495 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-217ee404-f71b-442c-8128-9c8a38b95f0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:24.956516 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ee404-f71b-442c-8128-9c8a38b95f0b service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:24.963463 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20150d4-ba37-4dfe-a332-349e6bc2a149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:24.964102 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20150d4-ba37-4dfe-a332-349e6bc2a149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:24.964413 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20150d4-ba37-4dfe-a332-349e6bc2a149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:24.964708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20150d4-ba37-4dfe-a332-349e6bc2a149 admin admin] PATH_INFO: `/v3/users/6113630b377f40eda2f8994ae69633ab` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:24.965852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20150d4-ba37-4dfe-a332-349e6bc2a149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6113630b377f40eda2f8994ae69633ab)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:24.969895 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20150d4-ba37-4dfe-a332-349e6bc2a149 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:25.242625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f20150d4-ba37-4dfe-a332-349e6bc2a149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6113630b377f40eda2f8994ae69633ab'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:25.249786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f20150d4-ba37-4dfe-a332-349e6bc2a149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6113630b377f40eda2f8994ae69633ab'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:25.326379 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f20150d4-ba37-4dfe-a332-349e6bc2a149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6113630b377f40eda2f8994ae69633ab'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:25.327010 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f20150d4-ba37-4dfe-a332-349e6bc2a149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6113630b377f40eda2f8994ae69633ab'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:25.332688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:25.341295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f20150d4-ba37-4dfe-a332-349e6bc2a149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6113630b377f40eda2f8994ae69633ab'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:25.365200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1366/2733] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:08:24 2022] DELETE /identity/v3/users/6113630b377f40eda2f8994ae69633ab => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:25.653613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7315bf8-537d-4938-9cc1-567f568f2e43 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:25.666363 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7315bf8-537d-4938-9cc1-567f568f2e43 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:25.745594 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:25.746251 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:25.746625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:25.746906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] PATH_INFO: `/v3/projects/a8e4d096ae8d4610a665f2c3f197d9b0` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:25.748119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8e4d096ae8d4610a665f2c3f197d9b0)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:25.750305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:25.871928 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8e4d096ae8d4610a665f2c3f197d9b0', 'request_id': 'req-8408e5b8-b922-4340-8461-0527b6f6e4d0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:25.880121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8e4d096ae8d4610a665f2c3f197d9b0', 'request_id': 'req-8408e5b8-b922-4340-8461-0527b6f6e4d0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:25.886885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8e4d096ae8d4610a665f2c3f197d9b0', 'request_id': 'req-8408e5b8-b922-4340-8461-0527b6f6e4d0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:25.981888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] The token cache is being invalidate because project a8e4d096ae8d4610a665f2c3f197d9b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:08:25.982181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:25.982867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8408e5b8-b922-4340-8461-0527b6f6e4d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:25.984346 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1368/2734] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:08:25 2022] DELETE /identity/v3/projects/a8e4d096ae8d4610a665f2c3f197d9b0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:25.989392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc36d11-e9a0-4976-836b-828ec6b20eb0 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:25.991117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc36d11-e9a0-4976-836b-828ec6b20eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:25.991117 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc36d11-e9a0-4976-836b-828ec6b20eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:26.010746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3fc36d11-e9a0-4976-836b-828ec6b20eb0 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:26.011979 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3fc36d11-e9a0-4976-836b-828ec6b20eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:26.073356 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1367/2735] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:26.081408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04341803-6d75-4cc6-b170-9b14bb26cb88 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:26.081697 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04341803-6d75-4cc6-b170-9b14bb26cb88 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:26.081974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04341803-6d75-4cc6-b170-9b14bb26cb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:26.103753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-04341803-6d75-4cc6-b170-9b14bb26cb88 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:26.104976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-04341803-6d75-4cc6-b170-9b14bb26cb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:26.117255 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1369/2736] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:26.122986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc36d11-e9a0-4976-836b-828ec6b20eb0 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:26.132096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc36d11-e9a0-4976-836b-828ec6b20eb0 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:26.137037 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afe7d0-0695-4621-9039-9765d7fa0b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:26.137713 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afe7d0-0695-4621-9039-9765d7fa0b4b admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:26.138064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afe7d0-0695-4621-9039-9765d7fa0b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:26.138429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afe7d0-0695-4621-9039-9765d7fa0b4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:26.139151 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afe7d0-0695-4621-9039-9765d7fa0b4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:08:26.139529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afe7d0-0695-4621-9039-9765d7fa0b4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:26.141214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afe7d0-0695-4621-9039-9765d7fa0b4b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:26.155754 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1368/2737] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:08:26.160492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04341803-6d75-4cc6-b170-9b14bb26cb88 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:26.170385 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04341803-6d75-4cc6-b170-9b14bb26cb88 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:26.175883 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd205b3-4552-4d2b-b042-4f98a54d6899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:26.176488 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd205b3-4552-4d2b-b042-4f98a54d6899 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:26.176831 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd205b3-4552-4d2b-b042-4f98a54d6899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:26.177062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd205b3-4552-4d2b-b042-4f98a54d6899 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:26.177715 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd205b3-4552-4d2b-b042-4f98a54d6899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:26.180750 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd205b3-4552-4d2b-b042-4f98a54d6899 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:26.327515 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1370/2738] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:08:26 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:26.332548 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afe7d0-0695-4621-9039-9765d7fa0b4b admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:26.342597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afe7d0-0695-4621-9039-9765d7fa0b4b admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:26.403931 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96888f6d-3b50-45be-9d8e-70f76649e6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:26.406481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96888f6d-3b50-45be-9d8e-70f76649e6cc admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:26.406764 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96888f6d-3b50-45be-9d8e-70f76649e6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:26.407038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96888f6d-3b50-45be-9d8e-70f76649e6cc admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:26.407674 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96888f6d-3b50-45be-9d8e-70f76649e6cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:26.410709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96888f6d-3b50-45be-9d8e-70f76649e6cc admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:26.514874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1369/2739] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:08:26 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:26.522375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd205b3-4552-4d2b-b042-4f98a54d6899 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:26.534906 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd205b3-4552-4d2b-b042-4f98a54d6899 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:26.552648 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10839cf2-f175-4633-9f7e-f05885173e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:26.552648 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10839cf2-f175-4633-9f7e-f05885173e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:26.552648 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10839cf2-f175-4633-9f7e-f05885173e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:26.552648 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10839cf2-f175-4633-9f7e-f05885173e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:26.553219 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10839cf2-f175-4633-9f7e-f05885173e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:26.554493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10839cf2-f175-4633-9f7e-f05885173e6c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:26.556197 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:26.566578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1371/2740] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:08:26.570719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96888f6d-3b50-45be-9d8e-70f76649e6cc admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:26.580956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96888f6d-3b50-45be-9d8e-70f76649e6cc admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:26.586075 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f16687-6b49-4fdd-b761-12d53e8d626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:26.586675 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f16687-6b49-4fdd-b761-12d53e8d626c admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:26.586956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f16687-6b49-4fdd-b761-12d53e8d626c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:26.587271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f16687-6b49-4fdd-b761-12d53e8d626c admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:26.587920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f16687-6b49-4fdd-b761-12d53e8d626c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:26.589571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f16687-6b49-4fdd-b761-12d53e8d626c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:26.602109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1370/2741] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:08:26.606306 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10839cf2-f175-4633-9f7e-f05885173e6c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:26.616393 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10839cf2-f175-4633-9f7e-f05885173e6c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:26.620945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ba443-3679-4fb0-a04f-aa8a73e17c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:26.621650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ba443-3679-4fb0-a04f-aa8a73e17c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:26.621919 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ba443-3679-4fb0-a04f-aa8a73e17c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:26.627581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ba443-3679-4fb0-a04f-aa8a73e17c4e admin admin] PATH_INFO: `/v3/projects/7424f67925334b2eb0a85057a44b9526/users/28a156e1ab8942438d51f7148ac75b85/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:26.637349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ba443-3679-4fb0-a04f-aa8a73e17c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7424f67925334b2eb0a85057a44b9526, user_id=28a156e1ab8942438d51f7148ac75b85, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:26.639969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ba443-3679-4fb0-a04f-aa8a73e17c4e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:26.772843 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1372/2742] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:08:26 2022] PUT /identity/v3/projects/7424f67925334b2eb0a85057a44b9526/users/28a156e1ab8942438d51f7148ac75b85/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:26.784325 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54fd5-6726-41a2-b0cf-339001ead495 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:26.784607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54fd5-6726-41a2-b0cf-339001ead495 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:26.784880 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b54fd5-6726-41a2-b0cf-339001ead495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:26.812135 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-67b54fd5-6726-41a2-b0cf-339001ead495 None None] MFA Rules not processed for user `28a156e1ab8942438d51f7148ac75b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:26.813368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-67b54fd5-6726-41a2-b0cf-339001ead495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:26.852749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:26.876023 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1371/2743] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:08:26.887030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ba443-3679-4fb0-a04f-aa8a73e17c4e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:26.888798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c56ba443-3679-4fb0-a04f-aa8a73e17c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:26.994173 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ba443-3679-4fb0-a04f-aa8a73e17c4e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:26.999859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd17cd7-ef79-46eb-a1aa-443d9e7fd8bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:27.000490 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd17cd7-ef79-46eb-a1aa-443d9e7fd8bc service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:27.000770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd17cd7-ef79-46eb-a1aa-443d9e7fd8bc service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:27.001058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd17cd7-ef79-46eb-a1aa-443d9e7fd8bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:27.009650 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd17cd7-ef79-46eb-a1aa-443d9e7fd8bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:27.011266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd17cd7-ef79-46eb-a1aa-443d9e7fd8bc service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:27.021595 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1373/2744] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:30.622008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b54fd5-6726-41a2-b0cf-339001ead495 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:30.707054 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b54fd5-6726-41a2-b0cf-339001ead495 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:30.711606 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:30.712264 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:30.712579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:30.712970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] PATH_INFO: `/v3/users/28a156e1ab8942438d51f7148ac75b85` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:30.714127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] RBAC: Authorizing `identity:delete_user(user_id=28a156e1ab8942438d51f7148ac75b85)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:30.716020 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:30.938506 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28a156e1ab8942438d51f7148ac75b85'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:30.949434 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28a156e1ab8942438d51f7148ac75b85'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:31.033524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28a156e1ab8942438d51f7148ac75b85'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:31.034169 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28a156e1ab8942438d51f7148ac75b85'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:31.048095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28a156e1ab8942438d51f7148ac75b85'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:31.055336 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1372/2745] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:08:30 2022] DELETE /identity/v3/users/28a156e1ab8942438d51f7148ac75b85 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:31.059915 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801a53-fa9e-4496-980a-e12856e7a124 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:31.060431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801a53-fa9e-4496-980a-e12856e7a124 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:31.060755 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801a53-fa9e-4496-980a-e12856e7a124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:31.081709 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1f801a53-fa9e-4496-980a-e12856e7a124 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:31.083082 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1f801a53-fa9e-4496-980a-e12856e7a124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:31.147505 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1374/2746] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:31.160439 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:31.162023 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:31.232327 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75adb1-15ea-4977-806e-4eb5c163b2db admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:31.235948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33389f-bbf4-4ed0-b0a7-83789db4dc40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:31.237337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33389f-bbf4-4ed0-b0a7-83789db4dc40 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:31.237662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33389f-bbf4-4ed0-b0a7-83789db4dc40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:31.237934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33389f-bbf4-4ed0-b0a7-83789db4dc40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:31.247939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33389f-bbf4-4ed0-b0a7-83789db4dc40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:31.249560 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33389f-bbf4-4ed0-b0a7-83789db4dc40 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:31.260518 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1373/2747] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:31.724973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801a53-fa9e-4496-980a-e12856e7a124 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:31.727473 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1f801a53-fa9e-4496-980a-e12856e7a124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:31.743760 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801a53-fa9e-4496-980a-e12856e7a124 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:31.748591 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:31.749293 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:31.749732 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:31.750200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] PATH_INFO: `/v3/projects/7424f67925334b2eb0a85057a44b9526` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:31.751328 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7424f67925334b2eb0a85057a44b9526)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:31.753796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:31.895090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:31.911432 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7424f67925334b2eb0a85057a44b9526', 'request_id': 'req-907d3f14-2e2a-48d9-b519-3657b494691f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:31.916708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7424f67925334b2eb0a85057a44b9526', 'request_id': 'req-907d3f14-2e2a-48d9-b519-3657b494691f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:31.923150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7424f67925334b2eb0a85057a44b9526', 'request_id': 'req-907d3f14-2e2a-48d9-b519-3657b494691f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:32.028917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] The token cache is being invalidate because project 7424f67925334b2eb0a85057a44b9526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:08:32.034185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:32.034185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-907d3f14-2e2a-48d9-b519-3657b494691f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:32.034185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1375/2748] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:08:31 2022] DELETE /identity/v3/projects/7424f67925334b2eb0a85057a44b9526 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:32.036567 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a5cfc-3370-46e6-8be9-b63b355e7159 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:32.037051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a5cfc-3370-46e6-8be9-b63b355e7159 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:32.037164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a5cfc-3370-46e6-8be9-b63b355e7159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:32.044585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:32.056926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-aa1a5cfc-3370-46e6-8be9-b63b355e7159 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:32.062857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-aa1a5cfc-3370-46e6-8be9-b63b355e7159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:32.118041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1374/2749] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:32.126231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a5d1e-de14-4054-ac2a-699498ab17d3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:32.127355 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a5d1e-de14-4054-ac2a-699498ab17d3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:32.127590 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a5d1e-de14-4054-ac2a-699498ab17d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:32.147455 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e22a5d1e-de14-4054-ac2a-699498ab17d3 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:32.148656 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e22a5d1e-de14-4054-ac2a-699498ab17d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:32.156564 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1376/2750] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:32.161213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a5cfc-3370-46e6-8be9-b63b355e7159 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:32.170750 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a5cfc-3370-46e6-8be9-b63b355e7159 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:32.176067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51fd71-b22b-4875-9948-d17c5d50079d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:32.176663 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51fd71-b22b-4875-9948-d17c5d50079d admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:32.176944 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51fd71-b22b-4875-9948-d17c5d50079d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:32.177244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51fd71-b22b-4875-9948-d17c5d50079d admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:32.177890 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc51fd71-b22b-4875-9948-d17c5d50079d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:08:32.178192 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc51fd71-b22b-4875-9948-d17c5d50079d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:32.179767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc51fd71-b22b-4875-9948-d17c5d50079d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:32.192893 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1375/2751] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:08:32.197406 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a5d1e-de14-4054-ac2a-699498ab17d3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:32.207061 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a5d1e-de14-4054-ac2a-699498ab17d3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:32.212184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562af832-e587-44f3-b448-b44f5bf2ac8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:32.212895 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562af832-e587-44f3-b448-b44f5bf2ac8e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:32.213296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562af832-e587-44f3-b448-b44f5bf2ac8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:32.213684 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562af832-e587-44f3-b448-b44f5bf2ac8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:32.214418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562af832-e587-44f3-b448-b44f5bf2ac8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:32.216392 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562af832-e587-44f3-b448-b44f5bf2ac8e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:32.334522 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1377/2752] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:08:32 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:32.338449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51fd71-b22b-4875-9948-d17c5d50079d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:32.348970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51fd71-b22b-4875-9948-d17c5d50079d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:32.419950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9def9c-6815-4a69-be3d-afcdab44bcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:32.420639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9def9c-6815-4a69-be3d-afcdab44bcfb admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:32.421051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9def9c-6815-4a69-be3d-afcdab44bcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:32.421431 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9def9c-6815-4a69-be3d-afcdab44bcfb admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:32.422186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9def9c-6815-4a69-be3d-afcdab44bcfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:32.424329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9def9c-6815-4a69-be3d-afcdab44bcfb admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:32.538314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1376/2753] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:08:32 2022] POST /identity/v3/users => generated 284 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:32.542438 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562af832-e587-44f3-b448-b44f5bf2ac8e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:32.553789 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562af832-e587-44f3-b448-b44f5bf2ac8e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:32.558562 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ddb7c-7be7-4483-a106-f3dd1d1ef24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:32.559107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ddb7c-7be7-4483-a106-f3dd1d1ef24a admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:32.559383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ddb7c-7be7-4483-a106-f3dd1d1ef24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:32.559644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ddb7c-7be7-4483-a106-f3dd1d1ef24a admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:32.560324 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ddb7c-7be7-4483-a106-f3dd1d1ef24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:32.561931 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ddb7c-7be7-4483-a106-f3dd1d1ef24a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:32.575198 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1378/2754] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:08:32.579408 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9def9c-6815-4a69-be3d-afcdab44bcfb admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:32.590355 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9def9c-6815-4a69-be3d-afcdab44bcfb admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:32.597563 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f3b4a-171b-43fc-8c26-62a60d791237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:32.598175 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f3b4a-171b-43fc-8c26-62a60d791237 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:32.598700 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f3b4a-171b-43fc-8c26-62a60d791237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:32.598826 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f3b4a-171b-43fc-8c26-62a60d791237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:32.599476 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f3b4a-171b-43fc-8c26-62a60d791237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:32.601676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f3b4a-171b-43fc-8c26-62a60d791237 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:32.614658 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1377/2755] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:08:32.619182 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ddb7c-7be7-4483-a106-f3dd1d1ef24a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:32.631075 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ddb7c-7be7-4483-a106-f3dd1d1ef24a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:32.636334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278218f4-4504-4dd7-bf32-5ed83d6802a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:32.637149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278218f4-4504-4dd7-bf32-5ed83d6802a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:32.637440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278218f4-4504-4dd7-bf32-5ed83d6802a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:32.637704 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278218f4-4504-4dd7-bf32-5ed83d6802a8 admin admin] PATH_INFO: `/v3/projects/17e5d7284208488ba3579dcf7a85a999/users/c16dca83c1df432896b2991463aa9cba/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:32.655204 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278218f4-4504-4dd7-bf32-5ed83d6802a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17e5d7284208488ba3579dcf7a85a999, user_id=c16dca83c1df432896b2991463aa9cba, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:32.658474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278218f4-4504-4dd7-bf32-5ed83d6802a8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:32.738481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1379/2756] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:08:32 2022] PUT /identity/v3/projects/17e5d7284208488ba3579dcf7a85a999/users/c16dca83c1df432896b2991463aa9cba/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:08:32.749952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65d38d-db06-44c0-a789-44321df8a33b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:32.749952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65d38d-db06-44c0-a789-44321df8a33b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:32.749952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65d38d-db06-44c0-a789-44321df8a33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:32.806048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-ff65d38d-db06-44c0-a789-44321df8a33b None None] MFA Rules not processed for user `c16dca83c1df432896b2991463aa9cba`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:32.807261 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ff65d38d-db06-44c0-a789-44321df8a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:32.879542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:32.887498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1378/2757] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:32.892351 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278218f4-4504-4dd7-bf32-5ed83d6802a8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:32.975258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278218f4-4504-4dd7-bf32-5ed83d6802a8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:32.981331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70668af-d1c6-491d-81f2-3bc3e69e76aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:32.982090 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70668af-d1c6-491d-81f2-3bc3e69e76aa admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:32.982377 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70668af-d1c6-491d-81f2-3bc3e69e76aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:32.982655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70668af-d1c6-491d-81f2-3bc3e69e76aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:32.983296 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70668af-d1c6-491d-81f2-3bc3e69e76aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:32.985241 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70668af-d1c6-491d-81f2-3bc3e69e76aa admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:33.122955 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1380/2758] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:08:32 2022] POST /identity/v3/projects => generated 369 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:33.127250 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65d38d-db06-44c0-a789-44321df8a33b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:33.138048 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65d38d-db06-44c0-a789-44321df8a33b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:33.172783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9bcce-3bca-47fa-a22b-a03dda4c2c20 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:33.173496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9bcce-3bca-47fa-a22b-a03dda4c2c20 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:33.174815 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9bcce-3bca-47fa-a22b-a03dda4c2c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:33.206541 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3620a-451b-40c9-aa18-003e632211cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:33.220559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3620a-451b-40c9-aa18-003e632211cb admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:33.220559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3620a-451b-40c9-aa18-003e632211cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:33.220559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3620a-451b-40c9-aa18-003e632211cb admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:33.220559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3620a-451b-40c9-aa18-003e632211cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:33.220559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3620a-451b-40c9-aa18-003e632211cb admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:33.235545 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-73a9bcce-3bca-47fa-a22b-a03dda4c2c20 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:33.237266 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-73a9bcce-3bca-47fa-a22b-a03dda4c2c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:33.258918 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:33.292697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1379/2759] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:08:33 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:08:33.296647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3620a-451b-40c9-aa18-003e632211cb admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:33.298098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1381/2760] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 10:08:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:33.312486 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3620a-451b-40c9-aa18-003e632211cb admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:33.318280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda44b8-db97-49b2-ade8-6da4eef79bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:33.318948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda44b8-db97-49b2-ade8-6da4eef79bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:33.319229 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda44b8-db97-49b2-ade8-6da4eef79bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:33.319590 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda44b8-db97-49b2-ade8-6da4eef79bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:33.320151 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda44b8-db97-49b2-ade8-6da4eef79bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:33.321785 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda44b8-db97-49b2-ade8-6da4eef79bb5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:33.335728 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1380/2761] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:08:33.337027 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9bcce-3bca-47fa-a22b-a03dda4c2c20 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:33.349380 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9bcce-3bca-47fa-a22b-a03dda4c2c20 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:33.354708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40108fb3-019a-4dff-9127-779ee550dc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:33.355384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40108fb3-019a-4dff-9127-779ee550dc36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:33.355796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40108fb3-019a-4dff-9127-779ee550dc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:33.356043 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40108fb3-019a-4dff-9127-779ee550dc36 admin admin] PATH_INFO: `/v3/projects/cbe9a21b2d6b447990fb306dbe2d1ba8/users/d06a8bbb8f8347819a06c7201abe2271/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:33.371303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40108fb3-019a-4dff-9127-779ee550dc36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe9a21b2d6b447990fb306dbe2d1ba8, user_id=d06a8bbb8f8347819a06c7201abe2271, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:33.373978 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40108fb3-019a-4dff-9127-779ee550dc36 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:33.456666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1382/2762] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:08:33 2022] PUT /identity/v3/projects/cbe9a21b2d6b447990fb306dbe2d1ba8/users/d06a8bbb8f8347819a06c7201abe2271/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:33.461064 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda44b8-db97-49b2-ade8-6da4eef79bb5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:33.468141 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40108fb3-019a-4dff-9127-779ee550dc36 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:33.470353 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-40108fb3-019a-4dff-9127-779ee550dc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:33.471676 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda44b8-db97-49b2-ade8-6da4eef79bb5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:33.531524 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ac014-12e8-4809-924b-080ff22332be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:33.532181 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ac014-12e8-4809-924b-080ff22332be admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:33.532471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ac014-12e8-4809-924b-080ff22332be admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:33.532747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ac014-12e8-4809-924b-080ff22332be admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:33.533547 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ac014-12e8-4809-924b-080ff22332be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:33.535132 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ac014-12e8-4809-924b-080ff22332be admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:33.546147 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40108fb3-019a-4dff-9127-779ee550dc36 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:33.546828 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1381/2763] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:08:33.551650 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ac014-12e8-4809-924b-080ff22332be admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:33.557200 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f61c07-5990-45bf-beb0-c988f0aeb8f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:33.558118 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f61c07-5990-45bf-beb0-c988f0aeb8f0 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:33.558407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f61c07-5990-45bf-beb0-c988f0aeb8f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:33.559099 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f61c07-5990-45bf-beb0-c988f0aeb8f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:33.562607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ac014-12e8-4809-924b-080ff22332be admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:33.568917 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb22a2-6310-4988-911b-cf1d4514e916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:33.569611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb22a2-6310-4988-911b-cf1d4514e916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:33.569892 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb22a2-6310-4988-911b-cf1d4514e916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:33.570162 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb22a2-6310-4988-911b-cf1d4514e916 admin admin] PATH_INFO: `/v3/projects/cbe9a21b2d6b447990fb306dbe2d1ba8/users/d06a8bbb8f8347819a06c7201abe2271/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:33.572196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb22a2-6310-4988-911b-cf1d4514e916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe9a21b2d6b447990fb306dbe2d1ba8, user_id=d06a8bbb8f8347819a06c7201abe2271, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:33.574850 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb22a2-6310-4988-911b-cf1d4514e916 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:33.593780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1382/2764] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:08:33 2022] PUT /identity/v3/projects/cbe9a21b2d6b447990fb306dbe2d1ba8/users/d06a8bbb8f8347819a06c7201abe2271/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:33.600643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00ac10-2b61-4535-8316-e6dab9866f9d None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:33.601300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00ac10-2b61-4535-8316-e6dab9866f9d None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:33.601300 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00ac10-2b61-4535-8316-e6dab9866f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:33.633005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-1c00ac10-2b61-4535-8316-e6dab9866f9d None None] MFA Rules not processed for user `d06a8bbb8f8347819a06c7201abe2271`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:33.634465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1c00ac10-2b61-4535-8316-e6dab9866f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:33.645389 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f61c07-5990-45bf-beb0-c988f0aeb8f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:33.647045 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f61c07-5990-45bf-beb0-c988f0aeb8f0 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:33.658286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1383/2765] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:08:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:08:33.695845 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:33.717041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1383/2766] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:08:33.728411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f61c07-5990-45bf-beb0-c988f0aeb8f0 service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:33.729841 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c7f61c07-5990-45bf-beb0-c988f0aeb8f0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:33.786867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:33.790720 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f61c07-5990-45bf-beb0-c988f0aeb8f0 service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:33.838571 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072453-dfc0-46ca-8215-adbce7ae75d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:33.839172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072453-dfc0-46ca-8215-adbce7ae75d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:33.839449 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072453-dfc0-46ca-8215-adbce7ae75d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:33.839722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072453-dfc0-46ca-8215-adbce7ae75d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:33.850729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072453-dfc0-46ca-8215-adbce7ae75d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:33.850729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072453-dfc0-46ca-8215-adbce7ae75d6 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:33.859481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1384/2767] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:34.448696 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bbe6a-255b-4898-a791-d720f05fcb0f None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:34.449185 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bbe6a-255b-4898-a791-d720f05fcb0f None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:34.449281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bbe6a-255b-4898-a791-d720f05fcb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:34.469605 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-355bbe6a-255b-4898-a791-d720f05fcb0f None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:34.471030 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-355bbe6a-255b-4898-a791-d720f05fcb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:34.527567 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1384/2768] 10.222.0.17 () {58 vars in 967 bytes} [Thu Aug 11 10:08:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:34.541037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072453-dfc0-46ca-8215-adbce7ae75d6 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:34.608131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072453-dfc0-46ca-8215-adbce7ae75d6 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:34.614068 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98f947-cb69-4441-82f8-b58806e58194 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:34.614691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98f947-cb69-4441-82f8-b58806e58194 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:34.614969 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98f947-cb69-4441-82f8-b58806e58194 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:34.615246 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98f947-cb69-4441-82f8-b58806e58194 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:34.623583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98f947-cb69-4441-82f8-b58806e58194 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:34.625170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98f947-cb69-4441-82f8-b58806e58194 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:34.634298 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1385/2769] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:08:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:08:35.321326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca454-16c2-4103-8cbb-7fd387de8d3b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:35.321615 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca454-16c2-4103-8cbb-7fd387de8d3b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:35.321893 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca454-16c2-4103-8cbb-7fd387de8d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:35.340948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-202ca454-16c2-4103-8cbb-7fd387de8d3b None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:35.342354 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-202ca454-16c2-4103-8cbb-7fd387de8d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:35.350625 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1385/2770] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 10:08:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:35.364807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98f947-cb69-4441-82f8-b58806e58194 service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:35.375451 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98f947-cb69-4441-82f8-b58806e58194 service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:35.382471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e6ee0-180e-41a1-8aef-2e5b6385abc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:35.383081 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e6ee0-180e-41a1-8aef-2e5b6385abc9 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:35.383396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e6ee0-180e-41a1-8aef-2e5b6385abc9 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:35.383675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e6ee0-180e-41a1-8aef-2e5b6385abc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:35.392468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e6ee0-180e-41a1-8aef-2e5b6385abc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:35.394427 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e6ee0-180e-41a1-8aef-2e5b6385abc9 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:35.402815 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1386/2771] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:08:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:08:36.167467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49724838-6139-4e79-878c-382196396e66 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:36.167947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49724838-6139-4e79-878c-382196396e66 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:36.168201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49724838-6139-4e79-878c-382196396e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:36.187668 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-49724838-6139-4e79-878c-382196396e66 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:36.188952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-49724838-6139-4e79-878c-382196396e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:36.197620 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1386/2772] 10.222.0.17 () {58 vars in 967 bytes} [Thu Aug 11 10:08:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:36.212114 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e6ee0-180e-41a1-8aef-2e5b6385abc9 service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:36.224058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e6ee0-180e-41a1-8aef-2e5b6385abc9 service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:36.230223 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a507fa-c584-45a5-8e2f-03c00a0c0510 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:36.230836 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a507fa-c584-45a5-8e2f-03c00a0c0510 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:36.231115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a507fa-c584-45a5-8e2f-03c00a0c0510 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:36.231386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a507fa-c584-45a5-8e2f-03c00a0c0510 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:36.242799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a507fa-c584-45a5-8e2f-03c00a0c0510 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:36.244774 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a507fa-c584-45a5-8e2f-03c00a0c0510 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:36.256425 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1387/2773] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:08:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:08:37.032312 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49724838-6139-4e79-878c-382196396e66 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:37.132749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49724838-6139-4e79-878c-382196396e66 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:37.140178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:37.140178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:37.140178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:37.140178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] PATH_INFO: `/v3/users/c16dca83c1df432896b2991463aa9cba` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:37.141394 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c16dca83c1df432896b2991463aa9cba)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:37.143641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:37.390504 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c16dca83c1df432896b2991463aa9cba'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:37.398691 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c16dca83c1df432896b2991463aa9cba'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:37.446563 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c16dca83c1df432896b2991463aa9cba'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:37.447295 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c16dca83c1df432896b2991463aa9cba'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:37.461900 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:37.462708 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c16dca83c1df432896b2991463aa9cba'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:37.470244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1387/2774] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:08:37 2022] DELETE /identity/v3/users/c16dca83c1df432896b2991463aa9cba => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:08:37.475809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a12a2-9f96-4602-96e7-7c7084c66aa8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:37.476352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a12a2-9f96-4602-96e7-7c7084c66aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:37.476691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a12a2-9f96-4602-96e7-7c7084c66aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:37.497770 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ca5a12a2-9f96-4602-96e7-7c7084c66aa8 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:37.499049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ca5a12a2-9f96-4602-96e7-7c7084c66aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:37.579996 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1388/2775] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:37.597596 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:37.599429 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:37.658470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8d299-62a3-4724-8af6-4693b0f4be7a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:37.662881 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d30b7b-84bc-43bc-a454-2c7b9f3e67ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:37.663500 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d30b7b-84bc-43bc-a454-2c7b9f3e67ad service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:37.663783 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d30b7b-84bc-43bc-a454-2c7b9f3e67ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:37.664056 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d30b7b-84bc-43bc-a454-2c7b9f3e67ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:37.675876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d30b7b-84bc-43bc-a454-2c7b9f3e67ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:37.677368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d30b7b-84bc-43bc-a454-2c7b9f3e67ad service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:37.690671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1388/2776] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:38.156551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a12a2-9f96-4602-96e7-7c7084c66aa8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:38.158066 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ca5a12a2-9f96-4602-96e7-7c7084c66aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:38.171114 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a12a2-9f96-4602-96e7-7c7084c66aa8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:38.187261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:38.194222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:38.194763 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:38.195423 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] PATH_INFO: `/v3/projects/17e5d7284208488ba3579dcf7a85a999` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:38.197102 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17e5d7284208488ba3579dcf7a85a999)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:38.199382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:38.206569 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:38.368235 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17e5d7284208488ba3579dcf7a85a999', 'request_id': 'req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:38.379655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17e5d7284208488ba3579dcf7a85a999', 'request_id': 'req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:38.389785 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17e5d7284208488ba3579dcf7a85a999', 'request_id': 'req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:38.487853 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] The token cache is being invalidate because project 17e5d7284208488ba3579dcf7a85a999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:08:38.490167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:38.496482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:38.496482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1389/2777] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:08:38 2022] DELETE /identity/v3/projects/17e5d7284208488ba3579dcf7a85a999 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:38.497008 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d30b7b-84bc-43bc-a454-2c7b9f3e67ad service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:38.508749 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d30b7b-84bc-43bc-a454-2c7b9f3e67ad service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:38.596962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:38.597894 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:38.599120 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:38.599120 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] PATH_INFO: `/v3/users/d06a8bbb8f8347819a06c7201abe2271` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:38.600337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d06a8bbb8f8347819a06c7201abe2271)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:38.602496 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:38.829727 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd06a8bbb8f8347819a06c7201abe2271'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:38.839301 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd06a8bbb8f8347819a06c7201abe2271'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:38.920905 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd06a8bbb8f8347819a06c7201abe2271'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:38.926158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd06a8bbb8f8347819a06c7201abe2271'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:38.939079 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd06a8bbb8f8347819a06c7201abe2271'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:38.946980 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1389/2778] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:08:38 2022] DELETE /identity/v3/users/d06a8bbb8f8347819a06c7201abe2271 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:39.326147 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:39.327753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:39.341395 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b597e4-7eb2-4e13-9e3c-ceaa8a065833 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:39.399560 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999b49f-7fc8-49b5-9c3f-9e07724e289b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:39.400189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b49f-7fc8-49b5-9c3f-9e07724e289b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:39.400469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b49f-7fc8-49b5-9c3f-9e07724e289b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:39.400740 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b49f-7fc8-49b5-9c3f-9e07724e289b admin admin] PATH_INFO: `/v3/users/f4a8f2f254eb4bf9bb8549ab66f46d22` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:39.401878 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999b49f-7fc8-49b5-9c3f-9e07724e289b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4a8f2f254eb4bf9bb8549ab66f46d22)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:39.403766 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999b49f-7fc8-49b5-9c3f-9e07724e289b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:39.443853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:39.445735 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:39.462877 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7f5ef-f47e-4f66-93ca-35ea7131c842 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:39.467774 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:39.472101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:39.472101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:39.472101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] PATH_INFO: `/v3/projects/cbe9a21b2d6b447990fb306dbe2d1ba8` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:39.472101 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbe9a21b2d6b447990fb306dbe2d1ba8)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:39.473007 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:39.485035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:39.591022 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9999b49f-7fc8-49b5-9c3f-9e07724e289b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4a8f2f254eb4bf9bb8549ab66f46d22'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:39.591675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9999b49f-7fc8-49b5-9c3f-9e07724e289b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4a8f2f254eb4bf9bb8549ab66f46d22'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:39.592281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9999b49f-7fc8-49b5-9c3f-9e07724e289b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4a8f2f254eb4bf9bb8549ab66f46d22'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:39.637341 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbe9a21b2d6b447990fb306dbe2d1ba8', 'request_id': 'req-f85214fe-0270-45ec-aa7b-c0eb5374185b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:39.644613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbe9a21b2d6b447990fb306dbe2d1ba8', 'request_id': 'req-f85214fe-0270-45ec-aa7b-c0eb5374185b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:39.651010 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbe9a21b2d6b447990fb306dbe2d1ba8', 'request_id': 'req-f85214fe-0270-45ec-aa7b-c0eb5374185b'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:39.675153 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9999b49f-7fc8-49b5-9c3f-9e07724e289b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4a8f2f254eb4bf9bb8549ab66f46d22'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:39.688486 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9999b49f-7fc8-49b5-9c3f-9e07724e289b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4a8f2f254eb4bf9bb8549ab66f46d22'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:39.708308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1390/2779] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:08:39 2022] DELETE /identity/v3/users/f4a8f2f254eb4bf9bb8549ab66f46d22 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:08:39.708308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19544c6c-a7ad-4087-b261-3787a69e0049 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:39.708308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19544c6c-a7ad-4087-b261-3787a69e0049 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:39.708308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19544c6c-a7ad-4087-b261-3787a69e0049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:39.727386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-19544c6c-a7ad-4087-b261-3787a69e0049 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:39.728644 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-19544c6c-a7ad-4087-b261-3787a69e0049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:39.737912 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:39.748314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] The token cache is being invalidate because project cbe9a21b2d6b447990fb306dbe2d1ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:08:39.748685 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:39.753991 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-f85214fe-0270-45ec-aa7b-c0eb5374185b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:39.753991 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1390/2780] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:08:39 2022] DELETE /identity/v3/projects/cbe9a21b2d6b447990fb306dbe2d1ba8 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:39.756747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fae0d-e5ed-480d-8289-c7fb3c75353a None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:39.757120 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fae0d-e5ed-480d-8289-c7fb3c75353a None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:39.757455 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fae0d-e5ed-480d-8289-c7fb3c75353a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:39.779829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-3c8fae0d-e5ed-480d-8289-c7fb3c75353a None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:39.781219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-3c8fae0d-e5ed-480d-8289-c7fb3c75353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:39.794631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1391/2781] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:08:39.804993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19544c6c-a7ad-4087-b261-3787a69e0049 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:39.806386 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-19544c6c-a7ad-4087-b261-3787a69e0049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:39.840448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1391/2782] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:39.848870 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f7e8b-bc2a-43ff-9640-5f66a1ae54eb None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:39.849224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f7e8b-bc2a-43ff-9640-5f66a1ae54eb None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:39.849499 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f7e8b-bc2a-43ff-9640-5f66a1ae54eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:39.864461 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19544c6c-a7ad-4087-b261-3787a69e0049 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:39.868307 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb2c6d-dda8-4e8d-bdb6-6e2d5f1894ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:39.868920 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb2c6d-dda8-4e8d-bdb6-6e2d5f1894ff service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:39.869051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-cb0f7e8b-bc2a-43ff-9640-5f66a1ae54eb None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:39.869232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb2c6d-dda8-4e8d-bdb6-6e2d5f1894ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:39.869508 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb2c6d-dda8-4e8d-bdb6-6e2d5f1894ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:39.870237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-cb0f7e8b-bc2a-43ff-9640-5f66a1ae54eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:39.882116 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1392/2783] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:08:39.882299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb2c6d-dda8-4e8d-bdb6-6e2d5f1894ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:39.882299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb2c6d-dda8-4e8d-bdb6-6e2d5f1894ff service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:39.883096 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f7e8b-bc2a-43ff-9640-5f66a1ae54eb None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:39.892554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1392/2784] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:08:39.898427 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f7e8b-bc2a-43ff-9640-5f66a1ae54eb None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:39.903209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2768b0e-c9fd-4587-ae04-2e781917115a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:39.903818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2768b0e-c9fd-4587-ae04-2e781917115a admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:39.904099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2768b0e-c9fd-4587-ae04-2e781917115a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:39.904377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2768b0e-c9fd-4587-ae04-2e781917115a admin admin] PATH_INFO: `/v3/domains` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:39.905047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2768b0e-c9fd-4587-ae04-2e781917115a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81639) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:08:39.905361 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2768b0e-c9fd-4587-ae04-2e781917115a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:39.906952 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2768b0e-c9fd-4587-ae04-2e781917115a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:39.919433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1393/2785] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:08:39.923712 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb2c6d-dda8-4e8d-bdb6-6e2d5f1894ff service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:39.933836 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb2c6d-dda8-4e8d-bdb6-6e2d5f1894ff service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:39.938467 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d030e20-9023-4c1f-9eb6-a0bd8d8cfbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:39.939069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d030e20-9023-4c1f-9eb6-a0bd8d8cfbc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:39.939349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d030e20-9023-4c1f-9eb6-a0bd8d8cfbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:39.939624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d030e20-9023-4c1f-9eb6-a0bd8d8cfbc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:39.941074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d030e20-9023-4c1f-9eb6-a0bd8d8cfbc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:39.943062 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d030e20-9023-4c1f-9eb6-a0bd8d8cfbc6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:40.036209 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1393/2786] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:08:39 2022] POST /identity/v3/projects => generated 361 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:40.040470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2768b0e-c9fd-4587-ae04-2e781917115a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:40.051114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2768b0e-c9fd-4587-ae04-2e781917115a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:40.110771 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16681fb0-2a6a-468c-9dd7-0f81e3b0b66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:40.111367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16681fb0-2a6a-468c-9dd7-0f81e3b0b66e admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:40.111652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16681fb0-2a6a-468c-9dd7-0f81e3b0b66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:40.111918 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16681fb0-2a6a-468c-9dd7-0f81e3b0b66e admin admin] PATH_INFO: `/v3/users` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:40.113105 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16681fb0-2a6a-468c-9dd7-0f81e3b0b66e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:40.115027 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16681fb0-2a6a-468c-9dd7-0f81e3b0b66e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:40.127626 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d030e20-9023-4c1f-9eb6-a0bd8d8cfbc6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:40.129649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4d030e20-9023-4c1f-9eb6-a0bd8d8cfbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:40.146276 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d030e20-9023-4c1f-9eb6-a0bd8d8cfbc6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:40.151151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:40.151791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:40.152143 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:40.152331 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] PATH_INFO: `/v3/projects/5a8afe85664248fd83617aab4e45c442` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:40.153589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8afe85664248fd83617aab4e45c442)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:40.155419 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:40.195444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1394/2787] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:08:40 2022] POST /identity/v3/users => generated 281 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:40.199641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16681fb0-2a6a-468c-9dd7-0f81e3b0b66e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:40.209656 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16681fb0-2a6a-468c-9dd7-0f81e3b0b66e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:40.214377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c13b5-95ee-40de-98df-d3949ad4aade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:40.217507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c13b5-95ee-40de-98df-d3949ad4aade admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:40.217507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c13b5-95ee-40de-98df-d3949ad4aade admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:40.217507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c13b5-95ee-40de-98df-d3949ad4aade admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:40.217507 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c13b5-95ee-40de-98df-d3949ad4aade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:40.218449 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c13b5-95ee-40de-98df-d3949ad4aade admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:40.226882 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:40.233304 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1395/2788] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:08:40.237410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c13b5-95ee-40de-98df-d3949ad4aade admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:40.247060 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c13b5-95ee-40de-98df-d3949ad4aade admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:40.253459 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8afe85664248fd83617aab4e45c442', 'request_id': 'req-2d11be31-a790-4c64-b032-6d483a8de6ba'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:40.258966 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8afe85664248fd83617aab4e45c442', 'request_id': 'req-2d11be31-a790-4c64-b032-6d483a8de6ba'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:40.265806 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8afe85664248fd83617aab4e45c442', 'request_id': 'req-2d11be31-a790-4c64-b032-6d483a8de6ba'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:40.304544 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9ae77-906a-4081-b357-543e1452daca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:40.305095 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9ae77-906a-4081-b357-543e1452daca admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:40.305376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9ae77-906a-4081-b357-543e1452daca admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:40.305643 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9ae77-906a-4081-b357-543e1452daca admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:40.306283 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9ae77-906a-4081-b357-543e1452daca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:40.307876 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9ae77-906a-4081-b357-543e1452daca admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:40.317393 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1396/2789] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:08:40.321561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9ae77-906a-4081-b357-543e1452daca admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:40.326052 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] The token cache is being invalidate because project 5a8afe85664248fd83617aab4e45c442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:08:40.326344 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:40.326992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d11be31-a790-4c64-b032-6d483a8de6ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:40.328495 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1394/2790] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:08:40 2022] DELETE /identity/v3/projects/5a8afe85664248fd83617aab4e45c442 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:40.331902 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9ae77-906a-4081-b357-543e1452daca admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:40.337219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f4cf4-f19e-4fc1-8e96-f0478e02c986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:40.337986 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f4cf4-f19e-4fc1-8e96-f0478e02c986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:40.338262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f4cf4-f19e-4fc1-8e96-f0478e02c986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:40.338533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f4cf4-f19e-4fc1-8e96-f0478e02c986 admin admin] PATH_INFO: `/v3/projects/e5ee555546bc431195b969dbf26df7b1/users/48ba0837abf94405818b76a985c0e70b/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:40.357337 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f4cf4-f19e-4fc1-8e96-f0478e02c986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ee555546bc431195b969dbf26df7b1, user_id=48ba0837abf94405818b76a985c0e70b, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:40.361130 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f4cf4-f19e-4fc1-8e96-f0478e02c986 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:40.442470 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1397/2791] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:08:40 2022] PUT /identity/v3/projects/e5ee555546bc431195b969dbf26df7b1/users/48ba0837abf94405818b76a985c0e70b/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 10:08:40.455170 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d4ec-2ad4-4e57-88dd-b59bcc94fb51 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:40.456373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d4ec-2ad4-4e57-88dd-b59bcc94fb51 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:40.457339 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d4ec-2ad4-4e57-88dd-b59bcc94fb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:40.490333 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f677d4ec-2ad4-4e57-88dd-b59bcc94fb51 None None] MFA Rules not processed for user `48ba0837abf94405818b76a985c0e70b`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:40.491630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f677d4ec-2ad4-4e57-88dd-b59bcc94fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:40.497910 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:40.566563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1395/2792] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:40.606146 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f4cf4-f19e-4fc1-8e96-f0478e02c986 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:40.607875 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8a1f4cf4-f19e-4fc1-8e96-f0478e02c986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:40.666628 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f4cf4-f19e-4fc1-8e96-f0478e02c986 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:40.670343 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff0a8f-11a2-4f96-b82c-ffb7bba875d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:40.670945 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff0a8f-11a2-4f96-b82c-ffb7bba875d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:40.671228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff0a8f-11a2-4f96-b82c-ffb7bba875d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:40.671503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff0a8f-11a2-4f96-b82c-ffb7bba875d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:40.679926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff0a8f-11a2-4f96-b82c-ffb7bba875d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:40.681389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff0a8f-11a2-4f96-b82c-ffb7bba875d7 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:40.690616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1398/2793] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:40.705482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677d4ec-2ad4-4e57-88dd-b59bcc94fb51 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:40.707622 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f677d4ec-2ad4-4e57-88dd-b59bcc94fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:40.771132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677d4ec-2ad4-4e57-88dd-b59bcc94fb51 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:40.775708 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:40.776334 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:40.776614 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:40.776888 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] PATH_INFO: `/v3/users/48ba0837abf94405818b76a985c0e70b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:40.778335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ba0837abf94405818b76a985c0e70b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:40.780274 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:40.994884 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ba0837abf94405818b76a985c0e70b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:40.995537 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ba0837abf94405818b76a985c0e70b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:40.996501 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ba0837abf94405818b76a985c0e70b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:41.051493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ba0837abf94405818b76a985c0e70b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:41.063872 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ba0837abf94405818b76a985c0e70b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:41.076049 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1396/2794] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:08:40 2022] DELETE /identity/v3/users/48ba0837abf94405818b76a985c0e70b => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:41.080597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf15da-b679-4cd3-9239-9d85cccef1a7 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:41.081533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf15da-b679-4cd3-9239-9d85cccef1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:41.081843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf15da-b679-4cd3-9239-9d85cccef1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:41.101168 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-e7bf15da-b679-4cd3-9239-9d85cccef1a7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:41.102394 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e7bf15da-b679-4cd3-9239-9d85cccef1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:41.160569 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1399/2795] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:41.170130 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:41.171578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:41.209245 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:41.236572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60316c-5194-4692-ac7f-5f2cdf4d93d7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:41.240503 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22502d01-d506-4c20-87a6-5a5d9f51c941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:41.241172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22502d01-d506-4c20-87a6-5a5d9f51c941 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:41.241453 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22502d01-d506-4c20-87a6-5a5d9f51c941 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:41.241728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22502d01-d506-4c20-87a6-5a5d9f51c941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:41.251909 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22502d01-d506-4c20-87a6-5a5d9f51c941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:41.252629 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22502d01-d506-4c20-87a6-5a5d9f51c941 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:41.262232 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1397/2796] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:41.670863 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf15da-b679-4cd3-9239-9d85cccef1a7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:41.672285 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-e7bf15da-b679-4cd3-9239-9d85cccef1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:41.684888 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf15da-b679-4cd3-9239-9d85cccef1a7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:41.689616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:41.690805 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:41.691076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:41.691348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] PATH_INFO: `/v3/projects/e5ee555546bc431195b969dbf26df7b1` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:41.692462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ee555546bc431195b969dbf26df7b1)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:41.694410 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:41.705285 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:41.833369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ee555546bc431195b969dbf26df7b1', 'request_id': 'req-6c3e170d-5799-41bc-9955-d7eb034300d0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:41.838856 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ee555546bc431195b969dbf26df7b1', 'request_id': 'req-6c3e170d-5799-41bc-9955-d7eb034300d0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:41.845629 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ee555546bc431195b969dbf26df7b1', 'request_id': 'req-6c3e170d-5799-41bc-9955-d7eb034300d0'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:41.958855 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] The token cache is being invalidate because project e5ee555546bc431195b969dbf26df7b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:08:41.959392 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:41.960092 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-6c3e170d-5799-41bc-9955-d7eb034300d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:41.961649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1400/2797] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:08:41 2022] DELETE /identity/v3/projects/e5ee555546bc431195b969dbf26df7b1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:41.966474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5936eea-dbc2-49d7-bd0a-048bfe2a0d1e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:41.966972 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5936eea-dbc2-49d7-bd0a-048bfe2a0d1e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:41.967069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5936eea-dbc2-49d7-bd0a-048bfe2a0d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:41.996551 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e5936eea-dbc2-49d7-bd0a-048bfe2a0d1e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:41.998923 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e5936eea-dbc2-49d7-bd0a-048bfe2a0d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:42.070783 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1398/2798] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:08:42.078763 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0801ec-e719-4d30-a68b-0fcd107650b7 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:42.079047 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0801ec-e719-4d30-a68b-0fcd107650b7 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:42.079320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0801ec-e719-4d30-a68b-0fcd107650b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:42.100083 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-0b0801ec-e719-4d30-a68b-0fcd107650b7 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:42.101351 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-0b0801ec-e719-4d30-a68b-0fcd107650b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:42.109950 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1401/2799] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:42.115281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5936eea-dbc2-49d7-bd0a-048bfe2a0d1e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:42.125396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5936eea-dbc2-49d7-bd0a-048bfe2a0d1e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:42.131131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a4da7-2898-4552-8e25-2a313d712fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:42.131754 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a4da7-2898-4552-8e25-2a313d712fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:42.132037 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a4da7-2898-4552-8e25-2a313d712fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:42.132692 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a4da7-2898-4552-8e25-2a313d712fe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:42.133326 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a4da7-2898-4552-8e25-2a313d712fe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:08:42.133635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a4da7-2898-4552-8e25-2a313d712fe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:42.135224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a4da7-2898-4552-8e25-2a313d712fe5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:42.148079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1399/2800] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:08:42.152448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0801ec-e719-4d30-a68b-0fcd107650b7 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:42.162966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0801ec-e719-4d30-a68b-0fcd107650b7 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:42.167995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a67f8-2ac1-4b6c-b07d-35605b303f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:42.168631 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a67f8-2ac1-4b6c-b07d-35605b303f44 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:42.168907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a67f8-2ac1-4b6c-b07d-35605b303f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:42.169209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a67f8-2ac1-4b6c-b07d-35605b303f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:42.169920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a67f8-2ac1-4b6c-b07d-35605b303f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:42.172462 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a67f8-2ac1-4b6c-b07d-35605b303f44 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:42.272148 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1402/2801] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:08:42 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:42.276356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a4da7-2898-4552-8e25-2a313d712fe5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:42.287957 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a4da7-2898-4552-8e25-2a313d712fe5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:42.350172 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48f17b-00a6-4f7b-b79b-fc135c90252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:42.350881 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48f17b-00a6-4f7b-b79b-fc135c90252e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:42.351162 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48f17b-00a6-4f7b-b79b-fc135c90252e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:42.351440 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48f17b-00a6-4f7b-b79b-fc135c90252e admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:42.352146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48f17b-00a6-4f7b-b79b-fc135c90252e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:42.354655 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48f17b-00a6-4f7b-b79b-fc135c90252e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:42.515628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1400/2802] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:08:42 2022] POST /identity/v3/users => generated 287 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:42.522542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a67f8-2ac1-4b6c-b07d-35605b303f44 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:42.533829 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a67f8-2ac1-4b6c-b07d-35605b303f44 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:42.540407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702951c-146b-4fa7-bdd2-1ff860245a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:42.541069 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702951c-146b-4fa7-bdd2-1ff860245a2a admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:42.541353 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702951c-146b-4fa7-bdd2-1ff860245a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:42.541626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702951c-146b-4fa7-bdd2-1ff860245a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:42.542271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702951c-146b-4fa7-bdd2-1ff860245a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:42.543866 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702951c-146b-4fa7-bdd2-1ff860245a2a admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:42.554847 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1403/2803] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:08:42.559024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48f17b-00a6-4f7b-b79b-fc135c90252e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:42.569759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48f17b-00a6-4f7b-b79b-fc135c90252e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:42.574478 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c35d53-648a-4773-af08-28b37493acac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:42.575079 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c35d53-648a-4773-af08-28b37493acac admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:42.575357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c35d53-648a-4773-af08-28b37493acac admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:42.575632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c35d53-648a-4773-af08-28b37493acac admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:42.576309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c35d53-648a-4773-af08-28b37493acac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:42.577930 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c35d53-648a-4773-af08-28b37493acac admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:42.589474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1401/2804] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:08:42.593810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702951c-146b-4fa7-bdd2-1ff860245a2a admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:42.606479 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702951c-146b-4fa7-bdd2-1ff860245a2a admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:42.611646 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aab87a-7de3-4aa9-a755-08447949aa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:42.612422 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aab87a-7de3-4aa9-a755-08447949aa3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:42.612930 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aab87a-7de3-4aa9-a755-08447949aa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:42.613484 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aab87a-7de3-4aa9-a755-08447949aa3c admin admin] PATH_INFO: `/v3/projects/46a68f5ebdaa4825836f3908ec50efc8/users/1cb3dd4b48c9468db2931fcf17a7b363/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:42.629657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aab87a-7de3-4aa9-a755-08447949aa3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a68f5ebdaa4825836f3908ec50efc8, user_id=1cb3dd4b48c9468db2931fcf17a7b363, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:42.632585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aab87a-7de3-4aa9-a755-08447949aa3c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:42.710277 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1404/2805] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:08:42 2022] PUT /identity/v3/projects/46a68f5ebdaa4825836f3908ec50efc8/users/1cb3dd4b48c9468db2931fcf17a7b363/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:42.722691 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31e797-23b7-46bb-9f94-06ac13962ba2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:42.722967 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31e797-23b7-46bb-9f94-06ac13962ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:42.723238 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31e797-23b7-46bb-9f94-06ac13962ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:42.753717 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6b31e797-23b7-46bb-9f94-06ac13962ba2 None None] MFA Rules not processed for user `1cb3dd4b48c9468db2931fcf17a7b363`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:42.754797 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6b31e797-23b7-46bb-9f94-06ac13962ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:42.785424 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:42.829976 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1402/2806] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:42.841407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aab87a-7de3-4aa9-a755-08447949aa3c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:42.843099 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-41aab87a-7de3-4aa9-a755-08447949aa3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:42.900777 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aab87a-7de3-4aa9-a755-08447949aa3c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:42.904445 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35bd41-7fb1-4d72-a54b-6322b404b1b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:42.905069 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35bd41-7fb1-4d72-a54b-6322b404b1b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:42.905348 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35bd41-7fb1-4d72-a54b-6322b404b1b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:42.905617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35bd41-7fb1-4d72-a54b-6322b404b1b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:42.913956 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35bd41-7fb1-4d72-a54b-6322b404b1b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:42.915373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35bd41-7fb1-4d72-a54b-6322b404b1b6 service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:42.923899 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1405/2807] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:46.952716 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31e797-23b7-46bb-9f94-06ac13962ba2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:47.019798 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31e797-23b7-46bb-9f94-06ac13962ba2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:47.024500 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:47.025158 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:47.025439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:47.025713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] PATH_INFO: `/v3/users/1cb3dd4b48c9468db2931fcf17a7b363` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:47.026825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cb3dd4b48c9468db2931fcf17a7b363)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:47.028701 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:47.293848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cb3dd4b48c9468db2931fcf17a7b363'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:47.294546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cb3dd4b48c9468db2931fcf17a7b363'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:47.295153 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cb3dd4b48c9468db2931fcf17a7b363'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:47.383119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cb3dd4b48c9468db2931fcf17a7b363'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:47.397121 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cb3dd4b48c9468db2931fcf17a7b363'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:47.410690 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1403/2808] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:08:46 2022] DELETE /identity/v3/users/1cb3dd4b48c9468db2931fcf17a7b363 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:47.415368 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3096a5-563d-4d73-b400-f6bc0b117e19 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:47.415854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3096a5-563d-4d73-b400-f6bc0b117e19 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:47.415854 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3096a5-563d-4d73-b400-f6bc0b117e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:47.435257 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-4e3096a5-563d-4d73-b400-f6bc0b117e19 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:47.436469 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4e3096a5-563d-4d73-b400-f6bc0b117e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:47.476636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:47.505262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1406/2809] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:47.515039 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:47.516550 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:47.564482 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:47.575393 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8b873-f4c8-41a0-8ae0-9e30d340f21b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:47.579058 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fde96-355f-486a-974c-bcbedbd11f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:47.579799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fde96-355f-486a-974c-bcbedbd11f5f service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:47.579937 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fde96-355f-486a-974c-bcbedbd11f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:47.580211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fde96-355f-486a-974c-bcbedbd11f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:47.589713 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fde96-355f-486a-974c-bcbedbd11f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:47.591135 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fde96-355f-486a-974c-bcbedbd11f5f service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:47.601700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1404/2810] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:48.209244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3096a5-563d-4d73-b400-f6bc0b117e19 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:48.209244 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4e3096a5-563d-4d73-b400-f6bc0b117e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:48.223233 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3096a5-563d-4d73-b400-f6bc0b117e19 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:48.228053 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:48.228780 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:48.229159 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:48.229503 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] PATH_INFO: `/v3/projects/46a68f5ebdaa4825836f3908ec50efc8` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:48.230715 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a68f5ebdaa4825836f3908ec50efc8)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:48.233543 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:48.356556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a68f5ebdaa4825836f3908ec50efc8', 'request_id': 'req-40997dbd-495c-442b-91f1-3fb311ef43b9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:48.362985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a68f5ebdaa4825836f3908ec50efc8', 'request_id': 'req-40997dbd-495c-442b-91f1-3fb311ef43b9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:48.369680 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a68f5ebdaa4825836f3908ec50efc8', 'request_id': 'req-40997dbd-495c-442b-91f1-3fb311ef43b9'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:48.457563 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] The token cache is being invalidate because project 46a68f5ebdaa4825836f3908ec50efc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:08:48.458209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:48.458396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-40997dbd-495c-442b-91f1-3fb311ef43b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:08:48.459925 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1407/2811] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:08:48 2022] DELETE /identity/v3/projects/46a68f5ebdaa4825836f3908ec50efc8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:48.464700 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da46cc-e105-46d3-8809-ee96aba28abb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:48.465174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da46cc-e105-46d3-8809-ee96aba28abb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:48.465431 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da46cc-e105-46d3-8809-ee96aba28abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:48.487640 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-33da46cc-e105-46d3-8809-ee96aba28abb None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:48.488875 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-33da46cc-e105-46d3-8809-ee96aba28abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:48.560155 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1405/2812] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:48.570438 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125662e7-cb46-406e-a3c6-4af0e637473e None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:48.570721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125662e7-cb46-406e-a3c6-4af0e637473e None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:48.570995 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125662e7-cb46-406e-a3c6-4af0e637473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:48.590548 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-125662e7-cb46-406e-a3c6-4af0e637473e None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:48.591769 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-125662e7-cb46-406e-a3c6-4af0e637473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:48.600177 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1408/2813] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:48.604493 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da46cc-e105-46d3-8809-ee96aba28abb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:48.614492 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da46cc-e105-46d3-8809-ee96aba28abb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:48.619002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b624327-d05c-4725-9e5b-c992c70a027c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:48.619598 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b624327-d05c-4725-9e5b-c992c70a027c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:48.619876 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b624327-d05c-4725-9e5b-c992c70a027c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:48.620150 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b624327-d05c-4725-9e5b-c992c70a027c admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:48.620792 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b624327-d05c-4725-9e5b-c992c70a027c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:08:48.621133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b624327-d05c-4725-9e5b-c992c70a027c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:48.622698 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b624327-d05c-4725-9e5b-c992c70a027c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:48.634688 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1406/2814] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 10:08:48.638852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125662e7-cb46-406e-a3c6-4af0e637473e None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:48.648641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125662e7-cb46-406e-a3c6-4af0e637473e None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:48.653317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5163b-0789-4d5b-a9db-a21b3c1737ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:48.653920 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5163b-0789-4d5b-a9db-a21b3c1737ce admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:48.654201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5163b-0789-4d5b-a9db-a21b3c1737ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:48.654475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5163b-0789-4d5b-a9db-a21b3c1737ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:48.655241 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5163b-0789-4d5b-a9db-a21b3c1737ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:48.657200 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5163b-0789-4d5b-a9db-a21b3c1737ce admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:48.802530 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1409/2815] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:08:48 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:48.806826 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b624327-d05c-4725-9e5b-c992c70a027c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:48.819076 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b624327-d05c-4725-9e5b-c992c70a027c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:48.878618 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221dfde1-a162-46e9-80cf-510ce0e509fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:48.879231 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221dfde1-a162-46e9-80cf-510ce0e509fb admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:48.879509 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221dfde1-a162-46e9-80cf-510ce0e509fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:48.879779 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221dfde1-a162-46e9-80cf-510ce0e509fb admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:48.880411 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221dfde1-a162-46e9-80cf-510ce0e509fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:48.882352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221dfde1-a162-46e9-80cf-510ce0e509fb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:48.966391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1407/2816] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:08:48 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:08:48.970745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5163b-0789-4d5b-a9db-a21b3c1737ce admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:48.982204 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5163b-0789-4d5b-a9db-a21b3c1737ce admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:48.987897 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccffbbc-e467-46c7-95e4-c55245a9a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:48.988617 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccffbbc-e467-46c7-95e4-c55245a9a6de admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:48.988898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccffbbc-e467-46c7-95e4-c55245a9a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:48.989201 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccffbbc-e467-46c7-95e4-c55245a9a6de admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:48.990104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccffbbc-e467-46c7-95e4-c55245a9a6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:48.991745 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccffbbc-e467-46c7-95e4-c55245a9a6de admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:49.003154 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1410/2817] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:08:49.007474 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221dfde1-a162-46e9-80cf-510ce0e509fb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:49.017760 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221dfde1-a162-46e9-80cf-510ce0e509fb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:49.022524 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4171088-194c-40e5-a566-2f446c38c3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:49.023120 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4171088-194c-40e5-a566-2f446c38c3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:49.023391 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4171088-194c-40e5-a566-2f446c38c3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:49.023660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4171088-194c-40e5-a566-2f446c38c3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:49.024297 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4171088-194c-40e5-a566-2f446c38c3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:49.025893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4171088-194c-40e5-a566-2f446c38c3a1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:49.036259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1408/2818] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:08:49.040292 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccffbbc-e467-46c7-95e4-c55245a9a6de admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:49.051078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccffbbc-e467-46c7-95e4-c55245a9a6de admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:49.055793 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9b3b7-d304-40f9-bdcf-8435e3e36537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:49.056376 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9b3b7-d304-40f9-bdcf-8435e3e36537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:49.056657 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9b3b7-d304-40f9-bdcf-8435e3e36537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:49.056962 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9b3b7-d304-40f9-bdcf-8435e3e36537 admin admin] PATH_INFO: `/v3/projects/08c8f4f458674aacb2cdaaade1fa1d9b/users/abd93b593cfe4bf28385ff1280431e77/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:49.071329 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9b3b7-d304-40f9-bdcf-8435e3e36537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c8f4f458674aacb2cdaaade1fa1d9b, user_id=abd93b593cfe4bf28385ff1280431e77, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:49.073999 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9b3b7-d304-40f9-bdcf-8435e3e36537 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:49.142199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1411/2819] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:08:49 2022] PUT /identity/v3/projects/08c8f4f458674aacb2cdaaade1fa1d9b/users/abd93b593cfe4bf28385ff1280431e77/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:08:49.152722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ce3ec-3bbe-49bf-b2c0-bcc493a4c670 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:49.153007 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ce3ec-3bbe-49bf-b2c0-bcc493a4c670 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:49.153300 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ce3ec-3bbe-49bf-b2c0-bcc493a4c670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:49.182056 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ca8ce3ec-3bbe-49bf-b2c0-bcc493a4c670 None None] MFA Rules not processed for user `abd93b593cfe4bf28385ff1280431e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:49.183286 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ca8ce3ec-3bbe-49bf-b2c0-bcc493a4c670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:49.225958 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:49.248989 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1409/2820] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:08:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:49.262127 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9b3b7-d304-40f9-bdcf-8435e3e36537 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:49.263791 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9ba9b3b7-d304-40f9-bdcf-8435e3e36537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:49.322584 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9b3b7-d304-40f9-bdcf-8435e3e36537 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:49.326138 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c111205-ba07-4871-9de9-3bb760de63da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:49.326746 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c111205-ba07-4871-9de9-3bb760de63da service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:49.327023 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c111205-ba07-4871-9de9-3bb760de63da service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:49.327289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c111205-ba07-4871-9de9-3bb760de63da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:49.336039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c111205-ba07-4871-9de9-3bb760de63da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:49.337498 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c111205-ba07-4871-9de9-3bb760de63da service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:49.341967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:08:49.347196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1412/2821] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:08:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:08:52.745675 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047bb5f-84eb-4092-a77c-78f674fe87d4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:52.746213 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047bb5f-84eb-4092-a77c-78f674fe87d4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:52.746554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047bb5f-84eb-4092-a77c-78f674fe87d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:52.768196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6047bb5f-84eb-4092-a77c-78f674fe87d4 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:08:52.769376 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6047bb5f-84eb-4092-a77c-78f674fe87d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:52.826131 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1410/2822] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Aug 11 10:08:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:08:52.839395 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c111205-ba07-4871-9de9-3bb760de63da service cinder] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:08:52.841220 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-9c111205-ba07-4871-9de9-3bb760de63da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:08:52.910155 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c111205-ba07-4871-9de9-3bb760de63da service cinder] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:08:52.915164 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d1bf8-c70b-4990-a417-163d03bd137b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:08:52.915758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d1bf8-c70b-4990-a417-163d03bd137b service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:08:52.916038 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d1bf8-c70b-4990-a417-163d03bd137b service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:08:52.916307 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d1bf8-c70b-4990-a417-163d03bd137b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:08:52.925320 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d1bf8-c70b-4990-a417-163d03bd137b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:08:52.926928 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d1bf8-c70b-4990-a417-163d03bd137b service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:08:52.934984 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1413/2823] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:08:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:09:02.819267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=33 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:02.931852 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=32 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:12.230814 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43ca3f-f397-461a-92ec-02d151baed35 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:12.231408 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43ca3f-f397-461a-92ec-02d151baed35 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:12.231606 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43ca3f-f397-461a-92ec-02d151baed35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:12.256024 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bc43ca3f-f397-461a-92ec-02d151baed35 None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:12.257418 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bc43ca3f-f397-461a-92ec-02d151baed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:12.265139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1411/2824] 10.222.0.17 () {58 vars in 967 bytes} [Thu Aug 11 10:09:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:09:12.282152 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d1bf8-c70b-4990-a417-163d03bd137b service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:12.300478 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d1bf8-c70b-4990-a417-163d03bd137b service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:12.308622 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36529ac0-bf43-4527-bf15-1c8b20914fac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:12.309440 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36529ac0-bf43-4527-bf15-1c8b20914fac service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:12.309842 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36529ac0-bf43-4527-bf15-1c8b20914fac service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:12.310213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36529ac0-bf43-4527-bf15-1c8b20914fac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:12.325756 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36529ac0-bf43-4527-bf15-1c8b20914fac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:12.328540 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36529ac0-bf43-4527-bf15-1c8b20914fac service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:12.343621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1414/2825] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:09:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:09:15.746681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43ca3f-f397-461a-92ec-02d151baed35 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:15.748167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bc43ca3f-f397-461a-92ec-02d151baed35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:15.819761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43ca3f-f397-461a-92ec-02d151baed35 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:15.824512 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45dd44-3576-441e-b425-4de5206f3870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:15.825201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45dd44-3576-441e-b425-4de5206f3870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:15.825485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45dd44-3576-441e-b425-4de5206f3870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:15.825761 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45dd44-3576-441e-b425-4de5206f3870 admin admin] PATH_INFO: `/v3/users/89cc9499c67e41929b2289cfb7dd4a74` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:15.826942 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45dd44-3576-441e-b425-4de5206f3870 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89cc9499c67e41929b2289cfb7dd4a74)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:15.828893 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45dd44-3576-441e-b425-4de5206f3870 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:16.015265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6c45dd44-3576-441e-b425-4de5206f3870 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89cc9499c67e41929b2289cfb7dd4a74'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:16.015929 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6c45dd44-3576-441e-b425-4de5206f3870 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89cc9499c67e41929b2289cfb7dd4a74'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:16.016536 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6c45dd44-3576-441e-b425-4de5206f3870 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89cc9499c67e41929b2289cfb7dd4a74'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:16.097107 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6c45dd44-3576-441e-b425-4de5206f3870 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89cc9499c67e41929b2289cfb7dd4a74'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:16.109975 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6c45dd44-3576-441e-b425-4de5206f3870 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89cc9499c67e41929b2289cfb7dd4a74'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:16.122649 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1412/2826] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:09:15 2022] DELETE /identity/v3/users/89cc9499c67e41929b2289cfb7dd4a74 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:16.355212 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36529ac0-bf43-4527-bf15-1c8b20914fac service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:16.373270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36529ac0-bf43-4527-bf15-1c8b20914fac service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:16.442734 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:16.444731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:16.445432 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:16.445758 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] PATH_INFO: `/v3/projects/a8690060da11478694b7110a10a00a21` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:16.446887 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8690060da11478694b7110a10a00a21)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:16.448797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:16.573305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8690060da11478694b7110a10a00a21', 'request_id': 'req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:16.578861 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8690060da11478694b7110a10a00a21', 'request_id': 'req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:16.585386 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8690060da11478694b7110a10a00a21', 'request_id': 'req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:16.667898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] The token cache is being invalidate because project a8690060da11478694b7110a10a00a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:09:16.668209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:16.673843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:16.673843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1415/2827] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:09:16 2022] DELETE /identity/v3/projects/a8690060da11478694b7110a10a00a21 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:19.237599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f824f-9f96-43e9-928d-c29aa50a679d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:19.238106 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f824f-9f96-43e9-928d-c29aa50a679d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:19.238359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f824f-9f96-43e9-928d-c29aa50a679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:19.269643 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c29f824f-9f96-43e9-928d-c29aa50a679d None None] MFA Rules not processed for user `813d8cff9d7d49d5995bfc272fee88f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:19.274819 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c29f824f-9f96-43e9-928d-c29aa50a679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:19.329151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1413/2828] 10.222.0.17 () {58 vars in 967 bytes} [Thu Aug 11 10:09:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:09:19.342025 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:19.343559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:19.437718 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f03c52-6160-4d7d-b4c8-e48dc4aef27c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:19.446103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b678571-70c7-4139-bc06-4c9e4313765d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915ee44218064af896020d008ff31957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:19.447228 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678571-70c7-4139-bc06-4c9e4313765d service swift] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:19.447814 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678571-70c7-4139-bc06-4c9e4313765d service swift] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:19.448318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b678571-70c7-4139-bc06-4c9e4313765d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:19.463448 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b678571-70c7-4139-bc06-4c9e4313765d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:19.466425 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b678571-70c7-4139-bc06-4c9e4313765d service swift] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:19.482321 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1416/2829] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Aug 11 10:09:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 10:09:28.448069 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f824f-9f96-43e9-928d-c29aa50a679d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:28.449994 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c29f824f-9f96-43e9-928d-c29aa50a679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:28.471849 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:28.534707 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f824f-9f96-43e9-928d-c29aa50a679d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:28.547616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:28.547616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:28.547616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:28.547616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] PATH_INFO: `/v3/users/1f3839edcb09423b8b4885dffbd87af9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:28.548133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3839edcb09423b8b4885dffbd87af9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:28.548133 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:28.938710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3839edcb09423b8b4885dffbd87af9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:28.939359 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3839edcb09423b8b4885dffbd87af9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:28.940128 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3839edcb09423b8b4885dffbd87af9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:29.022281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3839edcb09423b8b4885dffbd87af9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:29.035382 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3839edcb09423b8b4885dffbd87af9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:29.047410 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1414/2830] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:09:28 2022] DELETE /identity/v3/users/1f3839edcb09423b8b4885dffbd87af9 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:29.305871 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b678571-70c7-4139-bc06-4c9e4313765d service swift] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:29.317621 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b678571-70c7-4139-bc06-4c9e4313765d service swift] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:29.321709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:29.384837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:29.385580 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:29.385865 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:29.386137 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] PATH_INFO: `/v3/projects/93eee20443c14b739e06abe3eba34c44` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:29.387433 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=93eee20443c14b739e06abe3eba34c44)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:29.389670 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:29.497411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93eee20443c14b739e06abe3eba34c44', 'request_id': 'req-59082718-ae81-46b4-8c59-1233fc4acf1d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:29.502967 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93eee20443c14b739e06abe3eba34c44', 'request_id': 'req-59082718-ae81-46b4-8c59-1233fc4acf1d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:29.509067 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93eee20443c14b739e06abe3eba34c44', 'request_id': 'req-59082718-ae81-46b4-8c59-1233fc4acf1d'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:29.650460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] The token cache is being invalidate because project 93eee20443c14b739e06abe3eba34c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:09:29.650667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:29.651270 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:29.652767 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1417/2831] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:09:29 2022] DELETE /identity/v3/projects/93eee20443c14b739e06abe3eba34c44 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:29.656696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:29.658791 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:29.672303 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97257c5d-b56e-4629-9c77-e45ed7c28f5c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:29.732599 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3196009-5d72-4e07-ab7f-f06975ba7d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:29.733267 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196009-5d72-4e07-ab7f-f06975ba7d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:29.733549 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196009-5d72-4e07-ab7f-f06975ba7d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:29.733824 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3196009-5d72-4e07-ab7f-f06975ba7d48 admin admin] PATH_INFO: `/v3/users/dcf3c18f07704fbab2a34576e7d4d958` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:29.734945 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3196009-5d72-4e07-ab7f-f06975ba7d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf3c18f07704fbab2a34576e7d4d958)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:29.736984 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3196009-5d72-4e07-ab7f-f06975ba7d48 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:29.944897 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:29.950885 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b3196009-5d72-4e07-ab7f-f06975ba7d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf3c18f07704fbab2a34576e7d4d958'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:29.951570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b3196009-5d72-4e07-ab7f-f06975ba7d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf3c18f07704fbab2a34576e7d4d958'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:29.952185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b3196009-5d72-4e07-ab7f-f06975ba7d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf3c18f07704fbab2a34576e7d4d958'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:30.022625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b3196009-5d72-4e07-ab7f-f06975ba7d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf3c18f07704fbab2a34576e7d4d958'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:30.037441 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b3196009-5d72-4e07-ab7f-f06975ba7d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf3c18f07704fbab2a34576e7d4d958'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:30.051254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1415/2832] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:09:29 2022] DELETE /identity/v3/users/dcf3c18f07704fbab2a34576e7d4d958 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:30.314242 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:30.315919 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:30.329252 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59082718-ae81-46b4-8c59-1233fc4acf1d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:30.390460 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:30.391104 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:30.391381 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:30.391659 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] PATH_INFO: `/v3/projects/01ce5ccbff294fb1bef29e3d2785f254` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:30.393109 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ce5ccbff294fb1bef29e3d2785f254)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:30.394853 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:30.506352 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:30.513902 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ce5ccbff294fb1bef29e3d2785f254', 'request_id': 'req-1d85af12-ed98-4232-9e4f-69e4a95ecdff'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:30.520041 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ce5ccbff294fb1bef29e3d2785f254', 'request_id': 'req-1d85af12-ed98-4232-9e4f-69e4a95ecdff'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:30.526577 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ce5ccbff294fb1bef29e3d2785f254', 'request_id': 'req-1d85af12-ed98-4232-9e4f-69e4a95ecdff'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:30.701905 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] The token cache is being invalidate because project 01ce5ccbff294fb1bef29e3d2785f254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:09:30.702107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:30.702797 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-1d85af12-ed98-4232-9e4f-69e4a95ecdff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:30.704289 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1418/2833] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:09:30 2022] DELETE /identity/v3/projects/01ce5ccbff294fb1bef29e3d2785f254 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:30.710132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351678-ce36-4672-b784-e16a5a18ce09 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:30.710632 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351678-ce36-4672-b784-e16a5a18ce09 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:30.710731 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351678-ce36-4672-b784-e16a5a18ce09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:30.730917 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-24351678-ce36-4672-b784-e16a5a18ce09 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:30.732119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-24351678-ce36-4672-b784-e16a5a18ce09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:30.793768 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1416/2834] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:09:30.805057 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003fabb-9336-402e-a624-d44a690c2d8b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:30.805426 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003fabb-9336-402e-a624-d44a690c2d8b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:30.805796 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003fabb-9336-402e-a624-d44a690c2d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:30.826199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-2003fabb-9336-402e-a624-d44a690c2d8b None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:30.827464 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-2003fabb-9336-402e-a624-d44a690c2d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:30.837294 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1419/2835] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:09:30.841559 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24351678-ce36-4672-b784-e16a5a18ce09 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:30.852433 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24351678-ce36-4672-b784-e16a5a18ce09 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:30.857279 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc0d42-887b-4461-9bf5-13cea53ae8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:30.857887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc0d42-887b-4461-9bf5-13cea53ae8ab admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:30.858167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc0d42-887b-4461-9bf5-13cea53ae8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:30.858439 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc0d42-887b-4461-9bf5-13cea53ae8ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:30.859122 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc0d42-887b-4461-9bf5-13cea53ae8ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:09:30.859417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc0d42-887b-4461-9bf5-13cea53ae8ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:30.861065 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc0d42-887b-4461-9bf5-13cea53ae8ab admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:30.873067 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1417/2836] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:09:30.878125 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003fabb-9336-402e-a624-d44a690c2d8b None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:30.887652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003fabb-9336-402e-a624-d44a690c2d8b None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:30.892613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2f6f-5074-48e4-9457-6b7b2071c692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:30.893864 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2f6f-5074-48e4-9457-6b7b2071c692 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:30.894147 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2f6f-5074-48e4-9457-6b7b2071c692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:30.894412 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd2f6f-5074-48e4-9457-6b7b2071c692 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:30.900626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd2f6f-5074-48e4-9457-6b7b2071c692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:30.900626 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd2f6f-5074-48e4-9457-6b7b2071c692 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:31.041493 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1420/2837] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:09:30 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:09:31.045983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc0d42-887b-4461-9bf5-13cea53ae8ab admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:31.059132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc0d42-887b-4461-9bf5-13cea53ae8ab admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:31.116414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd90087-151f-4469-9acf-9267b5e1ed2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:31.117047 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd90087-151f-4469-9acf-9267b5e1ed2c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:31.117332 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd90087-151f-4469-9acf-9267b5e1ed2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:31.117613 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd90087-151f-4469-9acf-9267b5e1ed2c admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:31.118239 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd90087-151f-4469-9acf-9267b5e1ed2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:31.120140 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd90087-151f-4469-9acf-9267b5e1ed2c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:31.246261 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1418/2838] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:09:31 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:09:31.250697 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2f6f-5074-48e4-9457-6b7b2071c692 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:31.262382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd2f6f-5074-48e4-9457-6b7b2071c692 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:31.267059 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e415fea-4d9c-4672-a8c0-71dcb3e3c5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:31.267706 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e415fea-4d9c-4672-a8c0-71dcb3e3c5fb admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:31.267990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e415fea-4d9c-4672-a8c0-71dcb3e3c5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:31.268263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e415fea-4d9c-4672-a8c0-71dcb3e3c5fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:31.268964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e415fea-4d9c-4672-a8c0-71dcb3e3c5fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:31.270600 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e415fea-4d9c-4672-a8c0-71dcb3e3c5fb admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:31.281281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1421/2839] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:09:31.286443 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd90087-151f-4469-9acf-9267b5e1ed2c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:31.304807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd90087-151f-4469-9acf-9267b5e1ed2c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:31.313181 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd643a-3814-4996-bb72-1a413a267cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:31.313859 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd643a-3814-4996-bb72-1a413a267cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:31.314139 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd643a-3814-4996-bb72-1a413a267cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:31.314471 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd643a-3814-4996-bb72-1a413a267cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:31.315696 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd643a-3814-4996-bb72-1a413a267cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:31.317404 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd643a-3814-4996-bb72-1a413a267cd9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:31.328781 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1419/2840] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:09:31.333221 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e415fea-4d9c-4672-a8c0-71dcb3e3c5fb admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:31.343721 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e415fea-4d9c-4672-a8c0-71dcb3e3c5fb admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:31.351214 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b1c45-2c3f-4868-89cf-5b487e88ef2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:31.352196 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b1c45-2c3f-4868-89cf-5b487e88ef2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:31.356963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b1c45-2c3f-4868-89cf-5b487e88ef2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:31.356963 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b1c45-2c3f-4868-89cf-5b487e88ef2d admin admin] PATH_INFO: `/v3/projects/3ec3b6aa4cdb4187980e0fab84814b3c/users/f1a68eda0f5e4eec89510fdec0675f4f/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:31.370531 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b1c45-2c3f-4868-89cf-5b487e88ef2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec3b6aa4cdb4187980e0fab84814b3c, user_id=f1a68eda0f5e4eec89510fdec0675f4f, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:31.373224 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b1c45-2c3f-4868-89cf-5b487e88ef2d admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:31.467276 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1422/2841] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:09:31 2022] PUT /identity/v3/projects/3ec3b6aa4cdb4187980e0fab84814b3c/users/f1a68eda0f5e4eec89510fdec0675f4f/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:31.472880 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5066999-4954-4121-a2c9-458f5d4bb845 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:31.473189 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5066999-4954-4121-a2c9-458f5d4bb845 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:31.473463 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5066999-4954-4121-a2c9-458f5d4bb845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:31.492374 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f5066999-4954-4121-a2c9-458f5d4bb845 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:31.493601 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f5066999-4954-4121-a2c9-458f5d4bb845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:31.511485 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:31.554087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1420/2842] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:09:31.563762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b1c45-2c3f-4868-89cf-5b487e88ef2d admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:31.566256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1f7b1c45-2c3f-4868-89cf-5b487e88ef2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:31.628649 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b1c45-2c3f-4868-89cf-5b487e88ef2d admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:31.632298 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97479ab7-d16e-4257-a4d4-b0690d56932c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:31.632904 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97479ab7-d16e-4257-a4d4-b0690d56932c service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:31.633209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97479ab7-d16e-4257-a4d4-b0690d56932c service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:31.633475 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97479ab7-d16e-4257-a4d4-b0690d56932c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:31.643838 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97479ab7-d16e-4257-a4d4-b0690d56932c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:31.645073 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97479ab7-d16e-4257-a4d4-b0690d56932c service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:31.655390 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1423/2843] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:09:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:09:34.507873 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5066999-4954-4121-a2c9-458f5d4bb845 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:34.509489 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f5066999-4954-4121-a2c9-458f5d4bb845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:34.523222 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5066999-4954-4121-a2c9-458f5d4bb845 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:34.531375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:34.531375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:34.531375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:34.531375 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] PATH_INFO: `/v3/users/abd93b593cfe4bf28385ff1280431e77` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:34.531843 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd93b593cfe4bf28385ff1280431e77)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:34.532764 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:34.712771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd93b593cfe4bf28385ff1280431e77'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:34.712771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd93b593cfe4bf28385ff1280431e77'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:34.712771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd93b593cfe4bf28385ff1280431e77'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:34.799636 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd93b593cfe4bf28385ff1280431e77'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:34.818554 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd93b593cfe4bf28385ff1280431e77'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:34.834383 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1421/2844] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:09:34 2022] DELETE /identity/v3/users/abd93b593cfe4bf28385ff1280431e77 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:34.839156 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbc13c-e561-4c3b-8645-74fd9f711186 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:34.839730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbc13c-e561-4c3b-8645-74fd9f711186 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:34.839730 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbc13c-e561-4c3b-8645-74fd9f711186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:34.862157 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-91fbc13c-e561-4c3b-8645-74fd9f711186 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:34.864443 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-91fbc13c-e561-4c3b-8645-74fd9f711186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:34.936207 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:34.948133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1424/2845] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:09:34.958660 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:34.960848 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:35.034666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479f88-5d42-4e66-8d9e-e0f615a111b1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:35.038759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16247de-7f6b-4d32-92f2-d9c2096b94d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:35.039642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16247de-7f6b-4d32-92f2-d9c2096b94d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:35.040144 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16247de-7f6b-4d32-92f2-d9c2096b94d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:35.040563 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16247de-7f6b-4d32-92f2-d9c2096b94d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:35.051631 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16247de-7f6b-4d32-92f2-d9c2096b94d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:35.053258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16247de-7f6b-4d32-92f2-d9c2096b94d7 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:35.069171 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1422/2846] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:09:35.541857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbc13c-e561-4c3b-8645-74fd9f711186 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:35.541857 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-91fbc13c-e561-4c3b-8645-74fd9f711186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:35.560034 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbc13c-e561-4c3b-8645-74fd9f711186 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:35.567762 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:35.568452 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:35.568736 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:35.569039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] PATH_INFO: `/v3/projects/08c8f4f458674aacb2cdaaade1fa1d9b` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:35.570281 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c8f4f458674aacb2cdaaade1fa1d9b)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:35.572326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:35.771588 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c8f4f458674aacb2cdaaade1fa1d9b', 'request_id': 'req-a42c3aef-f581-4f48-807c-a2ff13ad6841'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:35.782063 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c8f4f458674aacb2cdaaade1fa1d9b', 'request_id': 'req-a42c3aef-f581-4f48-807c-a2ff13ad6841'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:35.788779 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c8f4f458674aacb2cdaaade1fa1d9b', 'request_id': 'req-a42c3aef-f581-4f48-807c-a2ff13ad6841'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:35.862946 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] The token cache is being invalidate because project 08c8f4f458674aacb2cdaaade1fa1d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81639) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:09:35.864253 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:35.865188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-a42c3aef-f581-4f48-807c-a2ff13ad6841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:35.870623 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1425/2847] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:09:35 2022] DELETE /identity/v3/projects/08c8f4f458674aacb2cdaaade1fa1d9b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:35.871748 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346974a-9c38-4da6-ba78-cfa98f462395 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:35.872202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346974a-9c38-4da6-ba78-cfa98f462395 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:35.872312 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346974a-9c38-4da6-ba78-cfa98f462395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:35.894978 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5346974a-9c38-4da6-ba78-cfa98f462395 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:35.895577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5346974a-9c38-4da6-ba78-cfa98f462395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:35.940541 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:35.989060 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1423/2848] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:09:36.070647 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14135a21-e887-4428-904b-51feb4d6f0d4 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:36.071186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14135a21-e887-4428-904b-51feb4d6f0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:36.071644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14135a21-e887-4428-904b-51feb4d6f0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:36.099237 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-14135a21-e887-4428-904b-51feb4d6f0d4 None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:36.100592 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-14135a21-e887-4428-904b-51feb4d6f0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:36.112719 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1426/2849] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:09:36.117641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346974a-9c38-4da6-ba78-cfa98f462395 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:36.130642 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346974a-9c38-4da6-ba78-cfa98f462395 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:36.136681 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb58608-b966-441f-b363-72c916caecd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:36.137434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb58608-b966-441f-b363-72c916caecd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:36.137776 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb58608-b966-441f-b363-72c916caecd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:36.138290 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb58608-b966-441f-b363-72c916caecd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:36.139184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb58608-b966-441f-b363-72c916caecd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 10:09:36.139685 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb58608-b966-441f-b363-72c916caecd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:36.141539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb58608-b966-441f-b363-72c916caecd3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:36.157373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1424/2850] 10.222.0.17 () {58 vars in 1153 bytes} [Thu Aug 11 10:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 10:09:36.163011 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14135a21-e887-4428-904b-51feb4d6f0d4 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:36.174735 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14135a21-e887-4428-904b-51feb4d6f0d4 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:36.182219 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc801b-53b5-427b-b550-f11ac8b6c297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:36.183142 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc801b-53b5-427b-b550-f11ac8b6c297 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:36.183561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc801b-53b5-427b-b550-f11ac8b6c297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:36.184035 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc801b-53b5-427b-b550-f11ac8b6c297 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:36.185118 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc801b-53b5-427b-b550-f11ac8b6c297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:36.195263 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc801b-53b5-427b-b550-f11ac8b6c297 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:36.281467 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1427/2851] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:09:36 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:09:36.286574 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb58608-b966-441f-b363-72c916caecd3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:36.301347 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb58608-b966-441f-b363-72c916caecd3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:36.374159 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8689b9-f50e-4fcd-9d42-64e9e477b32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:36.374935 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8689b9-f50e-4fcd-9d42-64e9e477b32e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:36.375953 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8689b9-f50e-4fcd-9d42-64e9e477b32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:36.376379 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8689b9-f50e-4fcd-9d42-64e9e477b32e admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:36.378624 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8689b9-f50e-4fcd-9d42-64e9e477b32e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:36.384520 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8689b9-f50e-4fcd-9d42-64e9e477b32e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:36.478992 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1425/2852] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:09:36 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 10:09:36.483262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc801b-53b5-427b-b550-f11ac8b6c297 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:36.498810 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc801b-53b5-427b-b550-f11ac8b6c297 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:36.504619 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c6ba8-28c5-4c6c-a00e-5974a8867ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:36.505641 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c6ba8-28c5-4c6c-a00e-5974a8867ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:36.506078 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c6ba8-28c5-4c6c-a00e-5974a8867ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:36.506447 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c6ba8-28c5-4c6c-a00e-5974a8867ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:36.507209 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c6ba8-28c5-4c6c-a00e-5974a8867ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:36.508947 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c6ba8-28c5-4c6c-a00e-5974a8867ab2 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:36.522748 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1428/2853] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:09:36.526951 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8689b9-f50e-4fcd-9d42-64e9e477b32e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:36.540254 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8689b9-f50e-4fcd-9d42-64e9e477b32e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:36.545396 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c43f9-0c5c-4005-910d-af10654933d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:36.546034 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c43f9-0c5c-4005-910d-af10654933d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:36.546468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c43f9-0c5c-4005-910d-af10654933d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:36.546749 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c43f9-0c5c-4005-910d-af10654933d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:36.547417 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c43f9-0c5c-4005-910d-af10654933d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:36.549570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c43f9-0c5c-4005-910d-af10654933d2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:36.562185 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1426/2854] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 10:09:36.567174 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c6ba8-28c5-4c6c-a00e-5974a8867ab2 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:36.578834 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c6ba8-28c5-4c6c-a00e-5974a8867ab2 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:36.586513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b805fbc-6994-4c3d-88b5-c95c17ccf5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:36.587268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b805fbc-6994-4c3d-88b5-c95c17ccf5db admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:36.587542 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b805fbc-6994-4c3d-88b5-c95c17ccf5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:36.587990 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b805fbc-6994-4c3d-88b5-c95c17ccf5db admin admin] PATH_INFO: `/v3/projects/99fd8347fa4243a38940a8b3a70a6b02/users/599019f4d63a49cd84755abedd9e8a3a/roles/bda5c75a326e4f5d9503960830898895` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:36.606031 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b805fbc-6994-4c3d-88b5-c95c17ccf5db admin admin] RBAC: Authorizing `identity:create_grant(project_id=99fd8347fa4243a38940a8b3a70a6b02, user_id=599019f4d63a49cd84755abedd9e8a3a, role_id=bda5c75a326e4f5d9503960830898895)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:36.609051 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b805fbc-6994-4c3d-88b5-c95c17ccf5db admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:36.686559 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1429/2855] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:09:36 2022] PUT /identity/v3/projects/99fd8347fa4243a38940a8b3a70a6b02/users/599019f4d63a49cd84755abedd9e8a3a/roles/bda5c75a326e4f5d9503960830898895 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:36.697628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0c51d-5fe8-4505-8e2c-2446c6bc603a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:36.697911 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0c51d-5fe8-4505-8e2c-2446c6bc603a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:36.698190 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0c51d-5fe8-4505-8e2c-2446c6bc603a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:36.732447 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-67e0c51d-5fe8-4505-8e2c-2446c6bc603a None None] MFA Rules not processed for user `599019f4d63a49cd84755abedd9e8a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:36.733710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-67e0c51d-5fe8-4505-8e2c-2446c6bc603a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:36.809866 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1427/2856] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:09:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 10:09:36.821926 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b805fbc-6994-4c3d-88b5-c95c17ccf5db admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:36.823699 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-8b805fbc-6994-4c3d-88b5-c95c17ccf5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:36.910407 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b805fbc-6994-4c3d-88b5-c95c17ccf5db admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:36.914629 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183afd10-99f4-4b77-8b76-945d3a2f0dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:36.915260 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183afd10-99f4-4b77-8b76-945d3a2f0dee service cinder] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:36.915770 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183afd10-99f4-4b77-8b76-945d3a2f0dee service cinder] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:36.916049 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183afd10-99f4-4b77-8b76-945d3a2f0dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:36.926481 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183afd10-99f4-4b77-8b76-945d3a2f0dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:36.928644 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183afd10-99f4-4b77-8b76-945d3a2f0dee service cinder] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:36.941044 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1430/2857] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:09:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:09:36.984084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0c51d-5fe8-4505-8e2c-2446c6bc603a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:37.029132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:37.062988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0c51d-5fe8-4505-8e2c-2446c6bc603a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:37.068149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb287d-cc9c-4697-8d05-324b81c96cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:37.068889 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb287d-cc9c-4697-8d05-324b81c96cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:37.071305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb287d-cc9c-4697-8d05-324b81c96cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:37.071305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb287d-cc9c-4697-8d05-324b81c96cdc admin admin] PATH_INFO: `/v3/users/599019f4d63a49cd84755abedd9e8a3a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:37.071305 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb287d-cc9c-4697-8d05-324b81c96cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=599019f4d63a49cd84755abedd9e8a3a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:37.072864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb287d-cc9c-4697-8d05-324b81c96cdc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:37.299652 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c1eb287d-cc9c-4697-8d05-324b81c96cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '599019f4d63a49cd84755abedd9e8a3a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:37.301373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c1eb287d-cc9c-4697-8d05-324b81c96cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '599019f4d63a49cd84755abedd9e8a3a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:37.303163 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c1eb287d-cc9c-4697-8d05-324b81c96cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '599019f4d63a49cd84755abedd9e8a3a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:37.384753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c1eb287d-cc9c-4697-8d05-324b81c96cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '599019f4d63a49cd84755abedd9e8a3a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:37.386747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a29383-7f4a-4217-aa32-47575c446b43 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:37.387029 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a29383-7f4a-4217-aa32-47575c446b43 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:37.387305 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a29383-7f4a-4217-aa32-47575c446b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:37.390817 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c1eb287d-cc9c-4697-8d05-324b81c96cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '599019f4d63a49cd84755abedd9e8a3a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:37.407555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1428/2858] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:09:36 2022] DELETE /identity/v3/users/599019f4d63a49cd84755abedd9e8a3a => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:37.409715 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7910a-8c42-4edb-bbfa-e1c6853fb20c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:37.409999 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7910a-8c42-4edb-bbfa-e1c6853fb20c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:37.410269 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7910a-8c42-4edb-bbfa-e1c6853fb20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:37.420314 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-77a29383-7f4a-4217-aa32-47575c446b43 None None] MFA Rules not processed for user `f1a68eda0f5e4eec89510fdec0675f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:37.421582 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-77a29383-7f4a-4217-aa32-47575c446b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:37.432546 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4ed7910a-8c42-4edb-bbfa-e1c6853fb20c None None] MFA Rules not processed for user `7936a8595af344cb984e181520011d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:37.434088 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4ed7910a-8c42-4edb-bbfa-e1c6853fb20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:37.447350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:37.511299 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1429/2859] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:09:37.522778 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7910a-8c42-4edb-bbfa-e1c6853fb20c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:37.524262 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4ed7910a-8c42-4edb-bbfa-e1c6853fb20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:37.538561 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1431/2860] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:09:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:09:37.542819 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a29383-7f4a-4217-aa32-47575c446b43 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:37.544265 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-77a29383-7f4a-4217-aa32-47575c446b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:37.558487 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a29383-7f4a-4217-aa32-47575c446b43 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:37.563269 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cb665-c9ba-4287-8c4b-5acd987d4380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:37.563957 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cb665-c9ba-4287-8c4b-5acd987d4380 admin admin] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:37.564534 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cb665-c9ba-4287-8c4b-5acd987d4380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:37.564815 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cb665-c9ba-4287-8c4b-5acd987d4380 admin admin] PATH_INFO: `/v3/projects` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:37.565627 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cb665-c9ba-4287-8c4b-5acd987d4380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:37.569686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cb665-c9ba-4287-8c4b-5acd987d4380 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:37.600000 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7910a-8c42-4edb-bbfa-e1c6853fb20c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:37.603908 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da09b4b-06c1-4341-b48f-5a3437cf4891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:37.604517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da09b4b-06c1-4341-b48f-5a3437cf4891 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:37.604796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da09b4b-06c1-4341-b48f-5a3437cf4891 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:37.605095 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da09b4b-06c1-4341-b48f-5a3437cf4891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:37.612578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:37.616184 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da09b4b-06c1-4341-b48f-5a3437cf4891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:37.617714 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da09b4b-06c1-4341-b48f-5a3437cf4891 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:37.628892 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1430/2861] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:09:37.654387 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1432/2862] 10.222.0.17 () {60 vars in 1153 bytes} [Thu Aug 11 10:09:37 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:09:37.658730 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da09b4b-06c1-4341-b48f-5a3437cf4891 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:37.670098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da09b4b-06c1-4341-b48f-5a3437cf4891 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:37.741339 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae951bce-1d05-43c7-a21e-830198bb401f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:37.741983 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae951bce-1d05-43c7-a21e-830198bb401f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:37.742258 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae951bce-1d05-43c7-a21e-830198bb401f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:37.742529 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae951bce-1d05-43c7-a21e-830198bb401f admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:37.743202 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae951bce-1d05-43c7-a21e-830198bb401f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:37.745115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae951bce-1d05-43c7-a21e-830198bb401f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:37.826849 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1431/2863] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Aug 11 10:09:37 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 10:09:37.828316 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cb665-c9ba-4287-8c4b-5acd987d4380 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:37.839964 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cb665-c9ba-4287-8c4b-5acd987d4380 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:37.845510 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b35f7e-30af-415c-8265-77d911d463f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:37.846206 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b35f7e-30af-415c-8265-77d911d463f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:37.846557 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b35f7e-30af-415c-8265-77d911d463f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:37.846901 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b35f7e-30af-415c-8265-77d911d463f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:37.847623 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b35f7e-30af-415c-8265-77d911d463f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:37.849707 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b35f7e-30af-415c-8265-77d911d463f9 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:37.865058 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1433/2864] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:09:37.867734 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae951bce-1d05-43c7-a21e-830198bb401f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:37.878555 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae951bce-1d05-43c7-a21e-830198bb401f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:37.883358 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8928691-a24e-4688-ac52-e7c1da165189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:37.884030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8928691-a24e-4688-ac52-e7c1da165189 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:37.884309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8928691-a24e-4688-ac52-e7c1da165189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:37.884581 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8928691-a24e-4688-ac52-e7c1da165189 admin admin] PATH_INFO: `/v3/projects/1c9863e9c4274e06a0d49902bcd26cec/users/94e93af44dc446cc80174b34d91ab490/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:37.901630 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8928691-a24e-4688-ac52-e7c1da165189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9863e9c4274e06a0d49902bcd26cec, user_id=94e93af44dc446cc80174b34d91ab490, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:37.904430 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8928691-a24e-4688-ac52-e7c1da165189 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:37.989635 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1432/2865] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:09:37 2022] PUT /identity/v3/projects/1c9863e9c4274e06a0d49902bcd26cec/users/94e93af44dc446cc80174b34d91ab490/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:37.994249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b35f7e-30af-415c-8265-77d911d463f9 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:38.006789 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b35f7e-30af-415c-8265-77d911d463f9 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:38.055572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8928691-a24e-4688-ac52-e7c1da165189 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:38.060664 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e8928691-a24e-4688-ac52-e7c1da165189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:38.072308 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866efaa-d865-4a7d-ac58-dbb2b39a31b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:38.072929 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866efaa-d865-4a7d-ac58-dbb2b39a31b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:38.073275 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866efaa-d865-4a7d-ac58-dbb2b39a31b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:38.073539 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866efaa-d865-4a7d-ac58-dbb2b39a31b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:38.074188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866efaa-d865-4a7d-ac58-dbb2b39a31b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:38.075786 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866efaa-d865-4a7d-ac58-dbb2b39a31b1 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:38.088318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1434/2866] 10.222.0.17 () {58 vars in 1122 bytes} [Thu Aug 11 10:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 10:09:38.092872 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866efaa-d865-4a7d-ac58-dbb2b39a31b1 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:38.105271 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866efaa-d865-4a7d-ac58-dbb2b39a31b1 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:38.110322 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dad6bb-e770-4df3-849c-71119ff52d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:38.111017 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dad6bb-e770-4df3-849c-71119ff52d65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:38.111317 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dad6bb-e770-4df3-849c-71119ff52d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:38.111611 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dad6bb-e770-4df3-849c-71119ff52d65 admin admin] PATH_INFO: `/v3/projects/1c9863e9c4274e06a0d49902bcd26cec/users/94e93af44dc446cc80174b34d91ab490/roles/caa057fb0c7c45258fbfb9dee30a22b3` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:38.113639 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dad6bb-e770-4df3-849c-71119ff52d65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9863e9c4274e06a0d49902bcd26cec, user_id=94e93af44dc446cc80174b34d91ab490, role_id=caa057fb0c7c45258fbfb9dee30a22b3)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:38.116268 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dad6bb-e770-4df3-849c-71119ff52d65 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:38.127672 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1435/2867] 10.222.0.17 () {60 vars in 1483 bytes} [Thu Aug 11 10:09:38 2022] PUT /identity/v3/projects/1c9863e9c4274e06a0d49902bcd26cec/users/94e93af44dc446cc80174b34d91ab490/roles/caa057fb0c7c45258fbfb9dee30a22b3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:38.167961 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8928691-a24e-4688-ac52-e7c1da165189 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:38.227407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:38.228042 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:38.228325 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:38.228605 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] PATH_INFO: `/v3/projects/99fd8347fa4243a38940a8b3a70a6b02` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:38.229759 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99fd8347fa4243a38940a8b3a70a6b02)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:38.231641 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:38.244366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:38.388799 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99fd8347fa4243a38940a8b3a70a6b02', 'request_id': 'req-78169018-34ed-4570-9046-04fa946d0434'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:38.395577 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99fd8347fa4243a38940a8b3a70a6b02', 'request_id': 'req-78169018-34ed-4570-9046-04fa946d0434'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:38.402074 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99fd8347fa4243a38940a8b3a70a6b02', 'request_id': 'req-78169018-34ed-4570-9046-04fa946d0434'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:38.481813 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] The token cache is being invalidate because project 99fd8347fa4243a38940a8b3a70a6b02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:09:38.482435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:38.483556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:09:38.485174 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1433/2868] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:09:38 2022] DELETE /identity/v3/projects/99fd8347fa4243a38940a8b3a70a6b02 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:09:43.844910 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08508925-91ff-450d-811b-e076b87d185b None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:43.846103 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08508925-91ff-450d-811b-e076b87d185b None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:43.846883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08508925-91ff-450d-811b-e076b87d185b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:43.879191 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-08508925-91ff-450d-811b-e076b87d185b None None] MFA Rules not processed for user `94e93af44dc446cc80174b34d91ab490`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:43.880444 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-08508925-91ff-450d-811b-e076b87d185b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:43.938015 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:43.969338 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1436/2869] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:09:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:09:43.988661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:09:43.988661 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:44.076763 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78169018-34ed-4570-9046-04fa946d0434 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:09:44.094084 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06928537-1488-4d8b-84b4-90f5438c0d63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:09:44.095012 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06928537-1488-4d8b-84b4-90f5438c0d63 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:44.095342 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06928537-1488-4d8b-84b4-90f5438c0d63 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:44.095625 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06928537-1488-4d8b-84b4-90f5438c0d63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:44.097867 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-06928537-1488-4d8b-84b4-90f5438c0d63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:44.162132 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06928537-1488-4d8b-84b4-90f5438c0d63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:09:44.163864 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06928537-1488-4d8b-84b4-90f5438c0d63 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:09:44.173167 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1434/2870] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:09:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:09:46.710610 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777d7f9-04e5-4150-8961-f8f566eeb8a6 None None] REQUEST_METHOD: `POST` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:09:46.711158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777d7f9-04e5-4150-8961-f8f566eeb8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:09:46.711158 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777d7f9-04e5-4150-8961-f8f566eeb8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:09:46.732246 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.auth.core [None req-a777d7f9-04e5-4150-8961-f8f566eeb8a6 None None] MFA Rules not processed for user `f1a68eda0f5e4eec89510fdec0675f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81639) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 10:09:46.733585 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a777d7f9-04e5-4150-8961-f8f566eeb8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:09:46.740809 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1437/2871] 10.222.0.17 () {58 vars in 958 bytes} [Thu Aug 11 10:09:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 10:09:54.168904 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:09:56.725533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=10 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:10:02.230281 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06928537-1488-4d8b-84b4-90f5438c0d63 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:02.231687 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-06928537-1488-4d8b-84b4-90f5438c0d63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:02.289341 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06928537-1488-4d8b-84b4-90f5438c0d63 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:02.292925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8d7de-bc43-4256-af1a-760bc654ec66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029bb992f9974eec83c0fe7f9fa58826', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:02.293556 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d7de-bc43-4256-af1a-760bc654ec66 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:02.293839 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d7de-bc43-4256-af1a-760bc654ec66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:02.294110 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d7de-bc43-4256-af1a-760bc654ec66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:02.295820 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-72e8d7de-bc43-4256-af1a-760bc654ec66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:02.309352 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8d7de-bc43-4256-af1a-760bc654ec66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:02.310752 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8d7de-bc43-4256-af1a-760bc654ec66 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:02.322137 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1435/2872] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:10:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:10:03.390465 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777d7f9-04e5-4150-8961-f8f566eeb8a6 None None] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:03.391938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a777d7f9-04e5-4150-8961-f8f566eeb8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:03.449731 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777d7f9-04e5-4150-8961-f8f566eeb8a6 None None] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:03.453304 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d10f16-7da3-4f39-a37e-5c6f043b2147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:03.453939 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d10f16-7da3-4f39-a37e-5c6f043b2147 service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:03.454249 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d10f16-7da3-4f39-a37e-5c6f043b2147 service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:03.454523 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d10f16-7da3-4f39-a37e-5c6f043b2147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:03.456389 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-50d10f16-7da3-4f39-a37e-5c6f043b2147 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:03.472005 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d10f16-7da3-4f39-a37e-5c6f043b2147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:03.473766 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d10f16-7da3-4f39-a37e-5c6f043b2147 service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:03.485545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1438/2873] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:10:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:10:12.316674 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=26 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:10:13.480370 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=26 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:10:20.206634 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8d7de-bc43-4256-af1a-760bc654ec66 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:20.208038 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-72e8d7de-bc43-4256-af1a-760bc654ec66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:20.219879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8d7de-bc43-4256-af1a-760bc654ec66 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:20.223638 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f1573-c17f-4008-b205-4f8915087c77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:20.224224 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f1573-c17f-4008-b205-4f8915087c77 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:20.224502 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f1573-c17f-4008-b205-4f8915087c77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:20.224771 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f1573-c17f-4008-b205-4f8915087c77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:20.233459 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f1573-c17f-4008-b205-4f8915087c77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:20.234847 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f1573-c17f-4008-b205-4f8915087c77 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:20.244690 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1436/2874] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:10:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:10:21.502556 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d10f16-7da3-4f39-a37e-5c6f043b2147 service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:21.504084 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-50d10f16-7da3-4f39-a37e-5c6f043b2147 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:21.519665 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d10f16-7da3-4f39-a37e-5c6f043b2147 service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:21.526357 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54073b-0fdf-4ba6-bd5e-5120fc667403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:21.526977 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54073b-0fdf-4ba6-bd5e-5120fc667403 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:21.527262 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54073b-0fdf-4ba6-bd5e-5120fc667403 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:21.527529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54073b-0fdf-4ba6-bd5e-5120fc667403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:21.529532 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-dd54073b-0fdf-4ba6-bd5e-5120fc667403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:21.544667 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54073b-0fdf-4ba6-bd5e-5120fc667403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:21.549280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54073b-0fdf-4ba6-bd5e-5120fc667403 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:21.557335 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1439/2875] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:10:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:10:23.029304 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f1573-c17f-4008-b205-4f8915087c77 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:23.039343 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f1573-c17f-4008-b205-4f8915087c77 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:23.043211 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d84b0-17ac-4f9e-abcd-5dccdc3d7798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:23.043816 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d84b0-17ac-4f9e-abcd-5dccdc3d7798 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:23.043938 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54073b-0fdf-4ba6-bd5e-5120fc667403 service nova] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:23.044357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d84b0-17ac-4f9e-abcd-5dccdc3d7798 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:23.044357 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7d84b0-17ac-4f9e-abcd-5dccdc3d7798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:23.046532 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-af7d84b0-17ac-4f9e-abcd-5dccdc3d7798 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:23.053382 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54073b-0fdf-4ba6-bd5e-5120fc667403 service nova] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:23.057046 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4295efe-548e-4f48-953a-efb1019a19ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:23.057671 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4295efe-548e-4f48-953a-efb1019a19ea service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:23.057948 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4295efe-548e-4f48-953a-efb1019a19ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:23.058218 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4295efe-548e-4f48-953a-efb1019a19ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:23.058384 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d84b0-17ac-4f9e-abcd-5dccdc3d7798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:23.059927 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7d84b0-17ac-4f9e-abcd-5dccdc3d7798 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:23.065985 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4295efe-548e-4f48-953a-efb1019a19ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:23.068245 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4295efe-548e-4f48-953a-efb1019a19ea service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:23.070320 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1437/2876] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:10:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:10:23.077535 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1440/2877] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:10:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 10:10:33.065435 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=12 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:10:33.074613 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=12 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:10:37.513925 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d84b0-17ac-4f9e-abcd-5dccdc3d7798 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:37.522753 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7d84b0-17ac-4f9e-abcd-5dccdc3d7798 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:37.526407 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8acd7-0fd4-42fc-bc7f-0b48c4711ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:37.527030 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8acd7-0fd4-42fc-bc7f-0b48c4711ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:37.527308 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8acd7-0fd4-42fc-bc7f-0b48c4711ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:37.527584 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8acd7-0fd4-42fc-bc7f-0b48c4711ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:37.529356 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-51e8acd7-0fd4-42fc-bc7f-0b48c4711ff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:37.592338 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8acd7-0fd4-42fc-bc7f-0b48c4711ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:37.593788 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e8acd7-0fd4-42fc-bc7f-0b48c4711ff9 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:37.604434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1438/2878] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:10:41.103350 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4295efe-548e-4f48-953a-efb1019a19ea service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:41.104837 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b4295efe-548e-4f48-953a-efb1019a19ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:41.175916 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4295efe-548e-4f48-953a-efb1019a19ea service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:41.181860 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b30212-d40d-48e8-9f32-6556a9384b7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:41.182471 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b30212-d40d-48e8-9f32-6556a9384b7f service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:41.182753 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b30212-d40d-48e8-9f32-6556a9384b7f service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:41.183028 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b30212-d40d-48e8-9f32-6556a9384b7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:41.184812 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-17b30212-d40d-48e8-9f32-6556a9384b7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:41.196982 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b30212-d40d-48e8-9f32-6556a9384b7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:41.198607 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b30212-d40d-48e8-9f32-6556a9384b7f service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:41.208684 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1441/2879] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:10:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:10:44.946589 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8acd7-0fd4-42fc-bc7f-0b48c4711ff9 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:44.958496 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8acd7-0fd4-42fc-bc7f-0b48c4711ff9 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:44.963676 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164a295-22aa-4ff7-a067-94c5791108cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:44.964413 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164a295-22aa-4ff7-a067-94c5791108cb service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:44.964777 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164a295-22aa-4ff7-a067-94c5791108cb service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:44.965151 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164a295-22aa-4ff7-a067-94c5791108cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:44.967156 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7164a295-22aa-4ff7-a067-94c5791108cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:44.980879 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164a295-22aa-4ff7-a067-94c5791108cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:44.982539 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164a295-22aa-4ff7-a067-94c5791108cb service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:44.998887 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1439/2880] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:10:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:10:45.056609 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b30212-d40d-48e8-9f32-6556a9384b7f service placement] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:45.058126 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-17b30212-d40d-48e8-9f32-6556a9384b7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:45.072652 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b30212-d40d-48e8-9f32-6556a9384b7f service placement] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:45.084211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:45.084211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:45.084211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:45.084211 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] PATH_INFO: `/v3/users/ee81d8d9cf994df68a15afe5cb9673c4` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:45.084688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee81d8d9cf994df68a15afe5cb9673c4)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:45.084688 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:45.246256 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee81d8d9cf994df68a15afe5cb9673c4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:45.253143 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee81d8d9cf994df68a15afe5cb9673c4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:45.324115 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee81d8d9cf994df68a15afe5cb9673c4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:45.324738 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee81d8d9cf994df68a15afe5cb9673c4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:45.337591 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee81d8d9cf994df68a15afe5cb9673c4'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:45.343377 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1442/2881] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:10:45 2022] DELETE /identity/v3/users/ee81d8d9cf994df68a15afe5cb9673c4 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:10:45.582729 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164a295-22aa-4ff7-a067-94c5791108cb service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:45.595063 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164a295-22aa-4ff7-a067-94c5791108cb service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:45.662728 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:45.663348 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:45.663628 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:45.663900 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] PATH_INFO: `/v3/projects/ea7b166d3468425ab368e369b4dd9989` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:45.665000 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea7b166d3468425ab368e369b4dd9989)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:45.666891 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:45.764294 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea7b166d3468425ab368e369b4dd9989', 'request_id': 'req-c212ea47-ed5c-4b66-9db0-78079ba91306'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:45.769807 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea7b166d3468425ab368e369b4dd9989', 'request_id': 'req-c212ea47-ed5c-4b66-9db0-78079ba91306'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:45.776057 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea7b166d3468425ab368e369b4dd9989', 'request_id': 'req-c212ea47-ed5c-4b66-9db0-78079ba91306'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:45.871765 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] The token cache is being invalidate because project ea7b166d3468425ab368e369b4dd9989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:10:45.872053 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:45.872666 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:45.874165 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1440/2882] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:10:45 2022] DELETE /identity/v3/projects/ea7b166d3468425ab368e369b4dd9989 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:10:45.878178 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:45.879583 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:45.893416 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff3eb4-d3d5-4892-965c-0cb1bac8c07c admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:45.950966 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:45.951616 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:45.951898 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:45.952173 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] PATH_INFO: `/v3/users/df3f4b89f25c4d06a4dfeb65c4fb52e7` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:45.953373 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df3f4b89f25c4d06a4dfeb65c4fb52e7)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:45.955318 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:45.958133 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:10:46.161636 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df3f4b89f25c4d06a4dfeb65c4fb52e7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:46.168703 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df3f4b89f25c4d06a4dfeb65c4fb52e7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:46.249907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df3f4b89f25c4d06a4dfeb65c4fb52e7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:46.250597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df3f4b89f25c4d06a4dfeb65c4fb52e7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:46.263630 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df3f4b89f25c4d06a4dfeb65c4fb52e7'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:46.269682 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1443/2883] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:10:45 2022] DELETE /identity/v3/users/df3f4b89f25c4d06a4dfeb65c4fb52e7 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:10:46.543117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:10:46.544580 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:10:46.558348 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212ea47-ed5c-4b66-9db0-78079ba91306 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:10:46.566739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:10:46.616517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:10:46.617249 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:10:46.617533 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:10:46.617809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] PATH_INFO: `/v3/projects/dd5c1df78c1d4190bda02186eedb3ff9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:10:46.618964 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd5c1df78c1d4190bda02186eedb3ff9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:10:46.620855 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:10:46.747410 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd5c1df78c1d4190bda02186eedb3ff9', 'request_id': 'req-f46c5d90-dd92-4361-95c8-28e1a997a9ff'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:46.752723 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd5c1df78c1d4190bda02186eedb3ff9', 'request_id': 'req-f46c5d90-dd92-4361-95c8-28e1a997a9ff'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:46.759607 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd5c1df78c1d4190bda02186eedb3ff9', 'request_id': 'req-f46c5d90-dd92-4361-95c8-28e1a997a9ff'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:46.855146 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] The token cache is being invalidate because project dd5c1df78c1d4190bda02186eedb3ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:10:46.855349 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:46.855993 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:10:46.857552 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1441/2884] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:10:46 2022] DELETE /identity/v3/projects/dd5c1df78c1d4190bda02186eedb3ff9 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:10:56.266198 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:10:56.852011 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:12:32.327513 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:12:32.328970 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:12:32.384597 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad143cd0-1da2-4946-9d13-bebc9f68daf5 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:12:32.430193 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870930f-062f-45a1-9c68-d73d53b3471f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:12:32.431386 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870930f-062f-45a1-9c68-d73d53b3471f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:12:32.431662 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870930f-062f-45a1-9c68-d73d53b3471f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:12:32.431933 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870930f-062f-45a1-9c68-d73d53b3471f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:12:32.433686 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-b870930f-062f-45a1-9c68-d73d53b3471f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:12:32.442747 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870930f-062f-45a1-9c68-d73d53b3471f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:12:32.444150 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b870930f-062f-45a1-9c68-d73d53b3471f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:12:32.452186 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1444/2885] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:12:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:12:32.864739 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:12:32.866201 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:12:32.879050 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c5d90-dd92-4361-95c8-28e1a997a9ff admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:12:32.882698 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec4c2f-91fd-42ec-87dd-1c8d656fa412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:12:32.883335 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec4c2f-91fd-42ec-87dd-1c8d656fa412 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:12:32.884149 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec4c2f-91fd-42ec-87dd-1c8d656fa412 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:12:32.884469 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec4c2f-91fd-42ec-87dd-1c8d656fa412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:12:32.886373 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-50ec4c2f-91fd-42ec-87dd-1c8d656fa412 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:12:32.897572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec4c2f-91fd-42ec-87dd-1c8d656fa412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:12:32.899021 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec4c2f-91fd-42ec-87dd-1c8d656fa412 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:12:32.908477 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1442/2886] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:12:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:12:32.983107 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870930f-062f-45a1-9c68-d73d53b3471f service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:12:32.991833 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b870930f-062f-45a1-9c68-d73d53b3471f service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:12:32.995533 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474325b-0e1e-4591-a422-18c78a3d4c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fc6c3d1ce8645808c22d1b59b15a100', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:12:32.996119 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474325b-0e1e-4591-a422-18c78a3d4c5f service neutron] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:12:32.996396 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474325b-0e1e-4591-a422-18c78a3d4c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:12:32.996661 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474325b-0e1e-4591-a422-18c78a3d4c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:12:32.998415 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a474325b-0e1e-4591-a422-18c78a3d4c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:12:33.067728 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474325b-0e1e-4591-a422-18c78a3d4c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:12:33.069187 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474325b-0e1e-4591-a422-18c78a3d4c5f service neutron] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:12:33.080409 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1445/2887] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:12:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:12:33.140809 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec4c2f-91fd-42ec-87dd-1c8d656fa412 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:12:33.142265 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-50ec4c2f-91fd-42ec-87dd-1c8d656fa412 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:12:33.212319 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec4c2f-91fd-42ec-87dd-1c8d656fa412 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:12:33.218115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11dfd3-8258-4621-892b-c87c4848f94a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:12:33.218796 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11dfd3-8258-4621-892b-c87c4848f94a service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:12:33.219126 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11dfd3-8258-4621-892b-c87c4848f94a service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:12:33.219412 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11dfd3-8258-4621-892b-c87c4848f94a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:12:33.221196 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2c11dfd3-8258-4621-892b-c87c4848f94a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:12:33.233000 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11dfd3-8258-4621-892b-c87c4848f94a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:12:33.234583 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11dfd3-8258-4621-892b-c87c4848f94a service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:12:33.245087 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1443/2888] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:12:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:12:35.514743 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474325b-0e1e-4591-a422-18c78a3d4c5f service neutron] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:12:35.516188 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-a474325b-0e1e-4591-a422-18c78a3d4c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:12:35.530925 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474325b-0e1e-4591-a422-18c78a3d4c5f service neutron] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:12:35.536315 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b97d5-788f-4e4c-b996-cc0f0102e200 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fa13a87ebe4b789ca1884e5aeb9237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:12:35.536907 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b97d5-788f-4e4c-b996-cc0f0102e200 service nova] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:12:35.537243 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b97d5-788f-4e4c-b996-cc0f0102e200 service nova] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:12:35.538689 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b97d5-788f-4e4c-b996-cc0f0102e200 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:12:35.539340 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-304b97d5-788f-4e4c-b996-cc0f0102e200 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:12:35.552076 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b97d5-788f-4e4c-b996-cc0f0102e200 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:12:35.553954 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b97d5-788f-4e4c-b996-cc0f0102e200 service nova] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:12:35.564411 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1446/2889] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:12:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:12:43.240259 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=38 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:12:45.560012 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=86 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:13:48.897117 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11dfd3-8258-4621-892b-c87c4848f94a service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:13:48.898572 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2c11dfd3-8258-4621-892b-c87c4848f94a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:13:48.912468 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11dfd3-8258-4621-892b-c87c4848f94a service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:13:48.917578 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1357627-1791-4342-8eb5-47032c2678c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:13:48.918264 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1357627-1791-4342-8eb5-47032c2678c5 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:13:48.918921 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1357627-1791-4342-8eb5-47032c2678c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:13:48.919260 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1357627-1791-4342-8eb5-47032c2678c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:13:48.921101 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d1357627-1791-4342-8eb5-47032c2678c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:13:48.937528 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1357627-1791-4342-8eb5-47032c2678c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:13:48.939115 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1357627-1791-4342-8eb5-47032c2678c5 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:13:48.949366 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1444/2890] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:13:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:13:58.945001 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:14:17.062934 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b97d5-788f-4e4c-b996-cc0f0102e200 service nova] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:14:17.064442 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-304b97d5-788f-4e4c-b996-cc0f0102e200 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:14:17.131199 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b97d5-788f-4e4c-b996-cc0f0102e200 service nova] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:14:17.136085 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:14:17.136709 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:14:17.137001 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:14:17.137331 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] PATH_INFO: `/v3/users/f1a68eda0f5e4eec89510fdec0675f4f` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:14:17.138450 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a68eda0f5e4eec89510fdec0675f4f)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:14:17.140349 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:14:17.375529 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a68eda0f5e4eec89510fdec0675f4f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:17.382818 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a68eda0f5e4eec89510fdec0675f4f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:17.501545 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a68eda0f5e4eec89510fdec0675f4f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:17.502163 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a68eda0f5e4eec89510fdec0675f4f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:17.516369 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a68eda0f5e4eec89510fdec0675f4f'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:17.523367 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1447/2891] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:14:17 2022] DELETE /identity/v3/users/f1a68eda0f5e4eec89510fdec0675f4f => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:14:17.813827 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1357627-1791-4342-8eb5-47032c2678c5 service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:14:17.822741 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1357627-1791-4342-8eb5-47032c2678c5 service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:14:17.883481 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:14:17.888309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:14:17.888309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:14:17.888309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] PATH_INFO: `/v3/projects/3ec3b6aa4cdb4187980e0fab84814b3c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:14:17.888309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec3b6aa4cdb4187980e0fab84814b3c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:14:17.888309 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:14:18.034434 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec3b6aa4cdb4187980e0fab84814b3c', 'request_id': 'req-96dae32f-57b5-4f44-966c-f68c7d5d33c0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:18.040002 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec3b6aa4cdb4187980e0fab84814b3c', 'request_id': 'req-96dae32f-57b5-4f44-966c-f68c7d5d33c0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:18.046243 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec3b6aa4cdb4187980e0fab84814b3c', 'request_id': 'req-96dae32f-57b5-4f44-966c-f68c7d5d33c0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:18.116699 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] The token cache is being invalidate because project 3ec3b6aa4cdb4187980e0fab84814b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:14:18.117119 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:18.117812 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:18.120832 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1445/2892] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:14:17 2022] DELETE /identity/v3/projects/3ec3b6aa4cdb4187980e0fab84814b3c => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:14:18.123366 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:14:18.125023 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:14:18.144144 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0600c-2725-4789-b270-b30aa8b18657 admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:14:18.206832 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:14:18.207567 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:14:18.207843 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:14:18.208114 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] PATH_INFO: `/v3/users/94e93af44dc446cc80174b34d91ab490` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:14:18.209285 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] RBAC: Authorizing `identity:delete_user(user_id=94e93af44dc446cc80174b34d91ab490)` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:14:18.212380 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:14:18.233280 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:14:18.491874 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94e93af44dc446cc80174b34d91ab490'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:18.498509 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94e93af44dc446cc80174b34d91ab490'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:18.599160 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94e93af44dc446cc80174b34d91ab490'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:18.599834 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94e93af44dc446cc80174b34d91ab490'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:18.612504 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.notifications [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94e93af44dc446cc80174b34d91ab490'} {{(pid=81639) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:18.618571 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1448/2893] 10.222.0.17 () {58 vars in 1224 bytes} [Thu Aug 11 10:14:18 2022] DELETE /identity/v3/users/94e93af44dc446cc80174b34d91ab490 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:14:18.951995 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:14:18.953570 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:14:18.968525 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dae32f-57b5-4f44-966c-f68c7d5d33c0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:14:19.028752 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7936a8595af344cb984e181520011d53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c055f43eb0f74f81a1a2e702ea1b7964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:14:19.029414 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:14:19.029710 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:14:19.029974 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] PATH_INFO: `/v3/projects/1c9863e9c4274e06a0d49902bcd26cec` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:14:19.031098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c9863e9c4274e06a0d49902bcd26cec)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:14:19.033098 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:14:19.184825 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:14:19.197517 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c9863e9c4274e06a0d49902bcd26cec', 'request_id': 'req-2ef43467-fa13-4f60-800c-f4a7d0d655a0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:19.202616 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c9863e9c4274e06a0d49902bcd26cec', 'request_id': 'req-2ef43467-fa13-4f60-800c-f4a7d0d655a0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:19.208988 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c9863e9c4274e06a0d49902bcd26cec', 'request_id': 'req-2ef43467-fa13-4f60-800c-f4a7d0d655a0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:19.298429 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] The token cache is being invalidate because project 1c9863e9c4274e06a0d49902bcd26cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 10:14:19.298719 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:19.299973 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ef43467-fa13-4f60-800c-f4a7d0d655a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 10:14:19.301615 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1446/2894] 10.222.0.17 () {58 vars in 1233 bytes} [Thu Aug 11 10:14:18 2022] DELETE /identity/v3/projects/1c9863e9c4274e06a0d49902bcd26cec => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 10:14:28.615536 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:14:29.421722 dv-cinder-815660-9-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 10:18:55.860842 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] Authenticating user token {{(pid=81639) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 10:18:55.862326 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:18:56.007039 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1da528-77fa-4fc3-8492-73e3b9d2a19e admin admin] Validating token access rules against request {{(pid=81639) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 10:18:56.070595 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769de4a-c78f-4149-8617-b14f680e630c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10ce125d46b545499de1fcf7072963e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca874f4c98ba4c499cf22ad790ac4c34', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81639) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 10:18:56.071213 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769de4a-c78f-4149-8617-b14f680e630c service placement] REQUEST_METHOD: `GET` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 10:18:56.071494 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769de4a-c78f-4149-8617-b14f680e630c service placement] SCRIPT_NAME: `/identity` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 10:18:56.071775 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769de4a-c78f-4149-8617-b14f680e630c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81639) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 10:18:56.073579 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.fernet_utils [None req-1769de4a-c78f-4149-8617-b14f680e630c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81639) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 10:18:56.085258 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769de4a-c78f-4149-8617-b14f680e630c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 10:18:56.086846 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769de4a-c78f-4149-8617-b14f680e630c service placement] RBAC: Authorization granted {{(pid=81639) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 10:18:56.096883 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: [pid: 81639|app: 0|req: 1449/2895] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Aug 11 10:18:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 10:19:06.092562 dv-cinder-815660-9-iscsi devstack@keystone.service[81639]: DEBUG dbcounter [-] [81639] Writing DB stats keystone:SELECT=89 {{(pid=81639) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}